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The  Editorial  Staff 


0 an  one  take  a step  forward  by 
taking  a step  backward?  The 
Defense  Management  Journal 
thinks  so. 

Periodically  every  organization 
should  step  back  and  attempt  to 
view  itself  objectively  to  see 
whether  its  course  is  still  true,  or 
whether  any  changes  or  adjust- 
ments are  required.  That’s  pre- 
cisely what  the  Journal  has  re- 
cently done;  as  a result,  some 
major  changes  are  planned  be- 
ginning in  January  1978. 

As  noted  in  our  last  issue,  the 
Journal  is  discontinuing  theme- 
oriented  issues.  In  their  place  will 
be  a modified  departmental  con- 
cept wherein  major  subject  areas 
such  as  manpower  and  personnel, 
logistics,  and  research  and  devel- 
opment will  be  represented  in 
every  issue.  This  revised  coverage 
will  provide  readers  with  a more 
diverse  selection  of  management 
articles  in  each  issue. 

Three  other  significant  changes 
are  also  programmed.  The  first 
concerns  frequency  of  publica- 
tion. Responding  to  a progressive 
demand  from  readers,  the  Journal 
staff  has  decided  to  commence 
bimonthly  publication.  The  ulti- 
mate goal  is  to  provide  readers  a 
timelier,  sooner-read  and  passed- 
on  Journal.  Naturally,  this  in- 
crease in  frequency  will  be 
accomplished  without  any  degra- 
dation of  product  quality. 

The  second  change  deals  with 


audience-level  focus.  Because  a 
large  portion  of  our  readership  now 
consists  of  middle  managers,  the 
Journal  will  include  a variety  of 
articles  targeted  to  that  level. 
There  will  be  articles  dealing  with 
theory  and  policy,  plus  practical 
management  operations  and  prac- 
tices. However,  this  will  infringe 
neither  the  specific  informational 
needs  nor  allegiance  of  senior 
management. 

The  third  change  relates  to  the 
reader’s  efficient  use  of  time.  The 
latest  Journal  readership  survey 
statistics  (see  the  April  1977 
Journal)  implied  that  the  higher 
the  grade,  the  less  time  spent  on 
reading.  Indeed,  we  recognize  that 
defense  managers  are  constantly 
faced  with  an  avalanche  of  written 
matter  and  not  enough  time  to 
read  all  of  it.  To  ameliorate  this 
situation  somewhat,  the  Journal 
will  provide  brief  article  abstracts 
as  part  of  the  table  of  contents. 
These  abstracts  will  enable  read- 
ers to  select  quickly  which  arti- 
cles they  want  to  read  in  full  and 
those  they  want  only  to  scan. 

All  of  these  innovations  are  a 
logical  response  to  the  changing 
demands  of  our  readers.  In  mak- 
ing them,  however,  the  Journal  is 
mindful  of  its  responsibility  to 
those  who  have  been  faithful 
readers  for  years  and  like  the 
magazine  the  way  it  is.  They  can 
rest  assured  that  the  Journal  will 
continue  to  address  the  key  man- 
agement trends,  issues,  policies, 


and  operations  that  concern  man- 
agers and  decisionmakers.  The 
Journal  will  continue  to  function 
as  an  active  and  viable  medium 
for  the  interchange  of  ideas  and 
techniques  for  management  im- 
provement. There  will  be  no  re- 
treat from  the  commitment  to 
timely,  in-depth  coverage,  nor  any 
less  concern  for  an  appealing  lay- 
out and  complementary  graphics. 

As  we  noted  in  our  report  on 
the  1976  readership  survey,  to 
thrive  as  a vital  and  stimulating 
publication,  we  need  to  hear  from 
all  segments  of  our  audience  from 
time  to  time.  Particularly  because 
we  are  making  the  aforemen- 
tioned changes,  your  criticisms, 
opinions,  and  insights  relative  to 
the  Journal’s  contents  are  all  the 
more  important.  In  broadening  our 
scope,  are  we  including  the  sub- 
jects you  want  to  read  about?  Or 
do  you  perhaps  disagree  with  a 
new  policy  directive?  The  Feed- 
back section  of  the  magazine  is 
the  place  to  construct  just  such  a 
dialogue  with  your  fellow  man- 
agers. But  your  input  is  obviously 
the  critical  ingredient. 

When  it  was  established  ten 
years  ago,  the  Journal  was  a 
unique  DoD  publication  treating 
management  improvement  with 
the  attention  it  deserves  and  por- 
traying management  as  an  art  and 
science.  This  is  still  the  role  the 
Journal  seeks  to  fulfill,  only  more 
often  in  the  coming  months  and 
years.  DMJ 
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Properly  applied,  test  and  evaluation  can  guide  managers 
to  the  best  system  at  the  most  affordable  price. 


W 

TT  hen  a separate  test  and 
evaluation  function  "was  formed 
within  the  Department  of  De- 
fense in  1971,  there  were  dire 
predictions  of  additional  man- 
agement layering,  a stretchout 
of  the  acquisition  cycle,  in- 
creased costs,  insignificant  bene- 
fits, and  other  adverse  effects. 
There  were  few  proponents. 

If  there  was  truly  no  value  to 
be  gained  by  this  move,  why  was 
T&E  formed  as  an  independent 
activity?  To  find  the  answer,  it 
is  necessary  to  go  back  to  the 
President’s  Blue  Ribbon  Defense 
Panel  Report  of  1970. 

The  Presidential  panel  made  a 
number  of  recommendations  de- 
signed to  improve  the  Defense 
Department’s  overall  operations. 
Particularly  concerned  with  the 
acquisition  process,  it  recom- 
mended abandoning  total  pack- 
age procurement  and  avoiding 
unnecessary  concurrency  in 
weapon  system  programs  be- 
tween the  development  and  pro- 
duction phases.  The  panel  also 
noted  the  operational  test  and 
evaluation  efforts  of  DoD  and 
the  military  services,  and  con- 
cluded that  the  results  were 
spotty  and  generally  ineffectual. 
Particularly  pertinent  to  this 
discussion  was  the  specific  rec- 
ommendation that  an  organiza- 
tion be  formed  that  would  report 


by  Rear  Adm.  L.  S.  Kollmorgen,  USN 
Assistant  Director 
Strategic  and  Support  Systems 
Test  and  Evaluation,  ODDR&E 


directly  to  the  Secretary  of  De- 
fense, would  be  at  a level  above 
the  services,  and  would  ensure 
that  operational  test  and  evalua- 
tion was  productively  accom- 
plished and  the  results  utilized 
effectively  in  the  acquisition 
process. 

In  arriving  at  this  recommen- 
dation, the  panel  reviewed  sev- 
eral weapon  system  case  histor- 
ies. The  M-16  rifle  was  perhaps 
the  most  widely  publicized  and 
typified  the  motivation  underly- 
ing the  panel’s  thinking. 

During  the  Vietnam  conflict, 
the  M-16’s  performance  had 
come  to  the  attention  of  Con- 
gress as  a result  of  letters  sent 
home  by  soldiers  complaining 
about  the  rifle’s  propensity  to 
jam.  The  reviewers  found  that 
the  M-16  had  gone  through  sev- 
eral modifications  subsequent  to 
approval  of  the  original  design 
without  a complete  field  test  of 
the  effect  of  those  modifications. 
The  original  design  had  received 
a complete  checkout  in  develop- 
ment tests.  However,  it  turned 
out  that  even  the  relatively  sim- 
ple design  of  the  M-16  was  sub- 
ject to  variations  in  perform- 
ance, because  design  changes  in 
one  portion  adversely  affected 
other  elements.  This  led  to  seri- 
ous malfunctions  in  combat. 

The  panel  observed  that  opera- 
tional testing  had  a direct  rela- 
tionship to  the  military  capabili- 
ties of  the  operating  forces. 
Therefore,  it  concluded  that  op- 
erational testing  required  con- 
tinued, high-level  emphasis. 


In  any  large  organization 
which  expends  large  sums  for 
acquiring  and  maintaining  sys- 
tems and  equipment,  there  is  a 
requirement  to  make  decisions 
that  take  into  consideration  al- 
ternative solutions.  The  selection 
of  an  alternative  is  facilitated  if 
adequate  T&E  is  conducted  and 
results  are  available  for  review. 

The  DoD  T&E  program  has 
two  primary  objectives: 

• Determination  of  essential 
operational  characteristics  of 
new  weapon  systems. 

o Evaluation  of  their  opera- 
tional effectiveness  and  suitabil- 
ity when  they  are  utilized  in  a 
realistic  environment. 

The  DoD  executives  who  make 
acquisition  decisions  must  con- 
sider the  relative  risk  that  some 
element  of  the  program  might 
produce  an  unintended  result 
which  could  adversely  affect  sys- 
tem effectiveness,  cost,  or  avail- 
ability for  deployment.  These  ex- 
ecutives have  come  to  depend  on 
the  results  and  assessments  of 
valid  tests  as  inputs  to  their  risk 
estimates. 

Besides  being  essential  for  de- 
cision making,  test  and  evalua- 
tion contributes  in  other  ways. 
Developmental  testing  and  eval- 
uation, for  example,  is  an  in- 
tegral part  of  the  process  by 
which  a system  design  is  con- 
verted into  hardware,  the  hard- 
ware tested,  deficiencies  noted, 
and  the  design  modified  as  nec- 
essary. This  process  is  iterated 
until  the  system  hardware  reach- 
es a design  configuration  con- 
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sidered  adequate  to  fulfill  speci- 
fied objectives.  DT&E  provides 
the  system  designers  with  essen- 
tial and  verifiable  feedback  in- 
formation that  would  be  difficult 
or  impossible  to  obtain  from 
purely  analytical  processes. 

As  noted  in  the  M-16  example, 
operational  testing  has  a direct 
relationship  to  the  military  capa- 
bilities of  operating  forces.  The 
greater  the  effort  made  to  iden- 
tify and  correct  any  operational 
deficiencies  discovered  before  the 
design  of  a system  is  frozen,  the 
greater  will  be  the  capabilities 
of  the  operating  forces  when  the 
new  weapon  system  is  deployed. 
Furthermore,  significant  cost 
and  time  benefits  may  be  realized 
if  major  system  deficiencies  are 
found  and  corrected  prior  to 
either  quantity  production  or  op- 
erational deployment.  Certainly 
it  is  less  disruptive  to  such  essen- 
tial activities  of  operating  forces 
as  training,  tactics  development, 
and  maintaining  a high  state  of 
operational  readiness,  if  exten- 
sive in-service  fault  correction 


and  retrofits  can  be  avoided. 

In  effecting  the  testing  policies 
set  forth  in  DoD  Directives 
5000.1,  “Acquisition  of  Defense 
Material,”  and  5000.3,  “Test  and 
Evaluation,”  it  is  sometimes  dif- 
ficult to  reconcile  the  guidance 
that  stems  from  different  points 
of  view.  For  example,  concern 
has  been  expressed  over  the  pon- 
derous nature  of  the  develop- 
mental process  and  the  potential 
impact  on  the  nation’s  military 
capability.  Those  voicing  this 
concern  view  test  and  evaluation 
as  a contributing  factor  to  what 
they  perceive  as  the  excessive 
length  of  the  acquisition  process, 
and  conclude  that  items  needed 
in  a hurry  must  be  removed  from 
the  process  in  order  to  get  the 
job  done  on  time. 

On  the  other  hand,  others  feel 
that  the  process  moves  too  fast 
and  are  disturbed  by  the  pres- 
sure applied  to  get  systems  into 
production  before  development 
has  been  completed. 

One  of  the  more  misinter- 
preted catchwords  used  in  con- 
nection with  test  and  evaluation 
is  “fly  before  buy,”  which  is 
often  associated  with  the  policies 
formulated  by  former  Deputy 
Secretary  of  Defense  David 
Packard.  It  has  been  variously 
interpreted,  and  is  perhaps  the 
cause  of  the  more  pejorative 
comments  connected  with  opera- 
tional test  and  evaluation.  This 
results  from  the  misperception 
of  initial  operational  testing  as  a 
major  hurdle  to  be  overcome 
prior  to  receiving  production  ap- 
proval. This  is  not  and  has  never 
been  the  intention  in  requiring 
adequate  OT&E  before  making 
the  major  production  decision. 

Reducing  Risk 

What  Mr.  Packard  was  advo- 
cating was  sound  program  plan- 


ning with  the  achievement  of  key 
development  objectives  validated 
through  test  and  evaluation,  and 
the  provision  of  adequate  time  in 
the  development  and  early  pro- 
duction period  to  allow  for  the 
correction  of  deficiencies  brought 
to  light  in  developmental  testing 
and  OT&E.  The  intention  was  to 
reduce  the  risk  of  unpleasant 
surprises  and  costly  corrections 
to  deployed  equipments  through 
increased  emphasis  on  adequate, 
as  well  as  operationally  relevant, 
testing. 

A successful  transition  from 
development  into  production  is 
not  characterized  by  an  abrupt 
termination  of  development  fol- 
lowed by  an  immediate  start  of 
production.  Rather,  it  is  a gradu- 
al process  by  which  the  produc- 
tion effort  is  phased  in  as  the 
development  effort  is  phased 
down,  but  not  necessarily  out. 
Testing  should  be  continued 
throughout  this  transition  period 
to  provide  input  to  the  produc- 
tion engineering  effort  and  con- 
tribute to  the  maturing  of  the 
weapon  system  design. 

The  key  is  selecting  the  appro- 
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priate  point  to  start  the  transi- 
tion. Ideally  this  would  be  when 
a system’s  design  and  technology 
are  well  in  hand  and  the  govern- 
ment will  benefit  from  the  most 
advantageous  application  of  in- 
vestment dollars.  In  short,  it  re- 
quires an  assessment  of  the  risk 
involved  in  initiating  production 
balanced  against  the  need  for  ac- 
quiring the  military  capability. 
Of  course,  this  implies  the  exer- 
cise of  sound  judgment. 

Unsatisfactory  Results 

The  policy  contained  in  DoDD 
5000.3  has  not  been  in  existence 
long  enough  to  have  influenced  a 
major  weapon  system  through- 
out the  acquisition  cycle ; that  is, 
from  initiation  through  produc- 
tion. It  may  be  useful,  however, 
to  review  some  experiences  that 
have  been  gained  during  the 
period  the  directive  has  been  in 
effect. 

For  a variety  of  reasons,  some 
programs  have  suffered  severe 
delays  or  have  produced  results 
far  short  of  the  intended  goals. 
The  causes  have  ranged  from  in- 
sufficient testing,  to  testing 


Firings  of  operational  POSEIDON  missiles  after  development  revealed  less- 
than-optimum  performance  levels. 
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which  discovered  deficiencies  too 
late,  to  adequate  testing  which 
pointed  out  shortcomings  but 
failed  to  influence  decisions  for 
adequate  corrective  action  before 
production  began. 

For  example,  in  the  Encapsu- 
lated Torpedo  (CAPTOR)  pro- 
gram, an  insufficient  number  of 
initial  test  articles  prevented 
early  determination,  within  ac- 
ceptable probability  bounds,  of 
the  causes  of  the  inadequate  per- 
formance observed  in  testing. 
The  result  was  a deferral  of  the 
major  production  decision  until 
follow-on  T&E  of  pilot  produc- 
tion items  could  be  conducted. 
This  is  sometimes  referred  to  as 
the  T&E  gap,  which  can  be 
avoided  by  planning  for  suffi- 
cient test  articles  and  adequate 
OT&E.  Provision  for  such  activ- 
ity is  necessary  to  bridge  the 
transition  from  development  to 
initial  production.  It  also  pro- 
vides for  a higher  degree  of  as- 
surance that,  lessons  learned 


from  the  testing  and  incorpo- 
rated into  the  early  production 
items  will  preclude  unpleasant 
surprises  in  production  systems. 

In  the  case  of  the  Navy’s 
HARPOON  anti-ship  missile,  the 
engineering  prototypes  complet- 
ed a successful  developmental 
test  program.  However,  opera- 
tional tests  of  pilot  production 
models  disclosed  failures  in  mis- 
sile components  which  rendered 
reliability  unacceptable.  The 
failures  developed  when  the 
transition  was  made  from  essen- 
tially handcrafting  missiles  to 
building  them  with  production 
materials,  tools,  processes,  and 
personnel.  The  initial,  low-rate 
production  was  maintained  until 
the  problems  were  resolved  and 
solutions  verified  by  additional 
operational  testing.  This  illus- 
trates the  desirability  of  early 
quality  assurance  tests,  the  need 
to  be  prepared  to  make  required 
changes  in  materials  and  proc- 
esses, and  the  value  of  a planned 
period  of  pilot  production  and 
testing  to  bridge  the  transition 
to  full  production. 

After  the  POSEIDON  missile 
was  deployed  in  1971,  scheduled 
firings  of  operational  missiles  in- 
dicated that  the  missile  was  not 
maintaining  the  performance 
level  demonstrated  during  devel- 
opmental tests.  Investigation  de- 
termined that  the  degradations 
in  performance  were  attribut- 
able primarily  to  the  lack  of  ade- 
quate component  testing  and 
screening  during  production 
rather  than  to  the  missile  design, 
which  had  been  verified  in  the 
developmental  program.  To  cor- 
rect this,  the  POSEIDON  Modi- 
fication Program  was  initiated 
at  a cost  of  more  than  $200  mil- 
lion, and  the  subsequent  per- 
formance of  the  missile  im- 
proved as  a result  of  changes 


made  to  production  processes. 
This  illustrates  the  validity  of 
continued  operational  testing  to 
ensure  that  no  degradation  in 
desired  performance  is  induced 
by  production  methods  or  the 
operational  environment. 

When  a program  has  provided 
for  adequate  testing  in  the  ap- 
propriate environments  and  test 
results  warn  of  problems,  the 
warning  must  be  heeded.  In  this 
way,  situations  similar  to  that 
of  the  Army’s  GAMA  GOAT 
might  be  avoided.  The  GAMA 
GOAT  is  a 1(4 -ton  truck  de- 
signed to  transport  cargo,  per- 
sonnel, and  weapons  both  on  and 
off  roads  in  military  operations. 

Even  though  prototype  testing 
in  1966  indicated  that  the  GAMA 
GOAT  would  not  be  suitable  for 
military  use  until  several  modifi- 
cations had  been  made  and  test- 
ed, the  vehicle  was  approved  for 
quantity  production.  Following 
1967  tests  in  which  reliability 
requirements  were  not  met,  a 
decision  was  made  to  procure 
more  than  15,000  vehicles. 

Testing  in  1969  on  preproduc- 
tion vehicles  revealed  mainte- 
nance and  durability  problems, 
and  in  1970  the  first  11  produc- 
tion vehicles  were  rejected  be- 
cause of  quality  control  defects. 
Production  continued,  however, 
and  GAMA  GOAT  was  deployed 
despite  high  maintenance  re- 
quirements, poor  durability,  and 
indications  that  the  Army  did 
not  have  a satisfactory  combat 
vehicle. 

Early  Correction 

As  mentioned  previously,  early 
detection  and  correction  of  defi- 
ciencies can  result  in  more  capa- 
bility in  the  operating  forces  and 
less  likelihood  of  costly  retrofit 
programs. 

A good  example  of  a recent 
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Although  prototype  and  preproduction  testing  of  the  Army’s  GAMA  GOAT  revealed  problems,  production  of  the  truck 
continued. 


case  where  T&E  revealed  a sys- 
tem deficiency  at  an  early  stage 
was  the  crew-escape  system  de- 
signed for  the  B-1  bomber.  Tests 
conducted  with  the  B-1  crew- 
escape  module  at  Holloman  Air 
Force  Base,  New  Mexico,  dem- 
onstrated that  the  system’s  aero- 
dynamic stability  could  not  be 
assured  at  speeds  above  300 
knots  per  hour.  Extensive  re- 
design involving  high  risk  and 
additional  cost  would  have  been 
necessary  to  provide  that  stabil- 
ity. Development  of  the  module 
was  therefore  terminated,  and 
an  off-the-shelf  ejection-seat  sys- 


tem is  being  modified  to  replace 
the  crew-escape  module. 

In  yet  another  example  of  the 
value  of  T&E  in  detecting  system 
defects,  the  Navy  discovered 
during  tests  of  the  prototype 
CH-53E  helicopter  that  the  main 
rotor  shaft  had  a low  fatigue 
life,  requiring  replacement  after 
only  230  hours  because  the  con- 
tractor had  underestimated  the 
dynamic  flight  loads  the  shaft 
would  have  to  sustain.  As  a re- 
sult of  these  tests,  a redesigned 
main  rotor  shaft  with  a fatigue 
life  of  11,000  hours  was  fabri- 
cated and  will  be  fully  tested 


and  subsequently  installed  in  the 
production  aircraft. 

During  the  early  1970’s,  em- 
phasis was  placed  almost  exclu- 
sively on  conducting  OT&E  just 
prior  to  the  major  production 
decision.  Defense  Systems  Ac- 
quisition Review  Council  Mile- 
stone III.  This  was  because  of 
the  large  procurement  funding 
commitment  usually  implicit  at 
Milestone  III.  Lately,  however, 
it  has  been  realized  that  earlier 
OT&E  may  be  beneficial  for  four 
reasons : 

• In  considering  system  con- 
cepts during  the  early  stages  of 
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a program,  it  is  necessary  to 
know  not  only  whether  they  are 
technically  feasible,  but  also 
whether  they  are  operationally 
viable  or  tactically  useful. 

• To  have  any  impact  on  the 
R&D  phase  of  a program,  OT&E 
must  be  conducted  earlier  than 
immediately  prior  to  DSARC  III, 
since  by  then  the  major  portion 
of  the  research  and  development 
resources  are  usually  expended. 

• The  earlier  in  the  develop- 
ment cycle  a deficiency  is  identi- 
fied, the  less  will  be  the  cost  of 
corrective  action. 

• The  successful  completion 
of  early  and  operationally  valid 
testing  reduces  the  risk  in  the 
long-lead  release  and  limited 
production  decisions. 

Indications  are  that  a better 
field  product  is  resulting  from 
emphasis  on  early  operational 
testing.  The  early  field  demon- 
stration of  desired  reliability 
goals  in  the  UTTAS  helicopter 
illustrates  this.  At  program  ini- 
tiation, DoD  placed  strong  em- 


phasis on  UTTAS  reliability  ob- 
jectives by  establishing  realistic 
preliminary  reliability  goals  and 
rules  for  reliability  measure- 
ments. These  goals  were  met  or 
exceeded  during  operational  test- 
ing by  Army  troops,  providing 
increased  confidence  that  the 
production  reliability  goal  would 
be  achieved  in  the  operational 
environment.  Currently,  DoDD 
5000.3  is  being  revised  to  reflect 
this  emphasis  on  earlier  opera- 
tional testing. 

Reliability 

Another  area  in  which  T&E  is 
now  making  a significant  contri- 
bution is  that  of  equipment  re- 
liability. This  is  not  a new  field 
of  interest ; the  services  have 
long  concentrated  on  it  and  have 
given  close  attention  to  achiev- 
ing improved  reliability  through 
proper  design  and  quality  con- 
trol in  the  production  process. 
T&E  plays  a major  role  in  the 
measurement  of  reliability  and 
isolation  of  failures.  The  actual 
correction  of  defects  is  accom- 
plished for  the  most  part  by  re- 
design of  the  equipment  or  im- 
proved quality  control  in  its 
manufacture. 

Laboratory  and  bench  tests  of 
components  and  assemblies  are 
the  mainstays  of  reliability  test- 
ing by  the  contractor,  and  form 
the  basis  for  proving  that  his 
equipment  complies  with  specifi- 
cations in  his  contract.  What 
these  tests  do  not  do  is  expose 
the  system  to  the  full  range  of 
operational  stresses,  the  harder 
handling  of  less-well-trained 
military  operational  and  mainte- 
nance personnel,  and  the  cumu- 
lative effects  of  harsh  field  con- 
ditions. 

If  these  reliability  weaknesses 
are  to  be  discovered  before  they 
adversely  affect  a large  number 


of  deployed  equipments,  the 
equipment  must  be  tested  under 
typical  operational  conditions. 
Therefore,  the  services  are  now 
required  to  specify  reliability  re- 
quirements in  operational  terms, 
measurable  under  operational 
conditions. 

A limiting  factor  in  operation- 
al testing  is  that  of  obtaining  a 
sufficient  number  of  hours  of 
system  operation  under  relevant 
conditions ; that  is,  full  system 
operation  in  an  actual  field  en- 
vironment. These  are  costly 
hours.  To  augment  them,  it  is 
necessary  to  ensure,  to  the  great- 
est extent  possible,  that  develop- 
mental testing  of  new  systems  is 
done  under  conditions  sufficient- 
ly realistic  that  the  failure  data 
can  be  combined  with  the  results 
of  operational  testing  to  give  a 
comprehensive,  yet  more  eco- 
nomical, method  of  evaluating 
weapon  system  reliability. 

How  Much  Is  Enough? 

Although  T&E  makes  a valu- 
able contribution  to  the  weapons 
acquisition  process,  it  is  import- 
ant to  ensure  that  testing  is  not 
conducted  unnecessarily.  There 
is  obviously  some  point  at  which 
the  incremental  value  of  infor- 
mation obtained  by  additional 
testing  is  not  worth  the  addition- 
al expense.  The  factors  which 
influence  the  judgment  about 
how  much  testing  is  needed  vary 
from  program  to  program. 

The  T&E  policy  changes  em- 
phasizing earlier  operational 
testing,  operationally  valid  reli- 
ability goals,  and  combined  de- 
velopmental and  opei’ational  test- 
ing are  expected  to  improve  the 
efficiency  of  the  acquisition  proc- 
ess in  general.  Specifically,  the 
greater  emphasis  on  earlier  op- 
erational exposure  to  systems 
under  test  should  provide  in- 
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The  Navy’s  HARPOON  anti-ship  missile  successfully  completed  developmental  testing,  but  operational  testing  dis 
closed  failures  in  missile  components. 


creased  confidence  that  weapon 
systems  so  tested  will  have  the 
desired  operational  characteris- 
tics upon  deployment.  In  addi- 
tion, the  weapon  design  teams 
should  profit  from  timely  feed- 
back of  the  results  of  operation- 
ally significant  tests. 

The  requirement  to  set  pre- 
liminary reliability  goals  speci- 
fied in  operationally  relevant 
terms  should  go  far  toward  pro- 
ducing equipment  that  meets 
the  users’  needs.  Hopefully,  reli- 
ance on  statistical  projections 
will  be  reduced. 

A closer  association  of  devel- 


opmental and  operational  testing 
should  provide  for  decreased 
costs  of  tests,  schedule  improve- 
ments, and  more  emphasis  on 
operationally  relevant  testing. 
As  a result,  decision  makers 
should  have  more  and  better  in- 
formation on  which  to  base  their 
judgments,  and  increased  knowl- 
edge of  any  inherent  risks. 

Of  course,  these  are  the  objec- 
tives desired.  Only  people  can 
make  things  happen,  and  only 
through  continued  dialogue  can 
we  ensure  that  the  true  intent 
and  objectives  of  DoD  policies 
are  understood.  Djvy 


REAR  ADMIRAL  L.  S.  KOLL- 
MORGEN,  USN,  is  the  Assistant 
Director,  Strategic  and  Support  Sys- 
tems Test  and  Evaluation,  in  the 
office  of  the  Director,  Defense  Re- 
search and  Engineering. 

Admiral  Kollmorgen’s  naval  career 
began  over  32  years  ago.  He  has  held 
numerous  assignments  of  increasing 
responsibility  including  duty  as  Dep- 
uty Chief  of  Naval  Operations  for  Air 
Warfare;  Special  Assistant  to  the  Di- 
rector, Navy  Program  Planning;  Com- 
mander of  Cecil  Field  Naval  Air  Sta- 
tion; and  prior  to  his  current  assign- 
ment, he  was  appointed  as  Military 
Assistant  to  President  Ford. 

Admiral  Kollmorgen  is  a graduate 
of  the  U.  S.  Naval  Academy. 
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Tailoring 

Specifications  and  Standards 
To  Today's  Needs 


Specifications  and  standards  have  too  often  been  treated  more  as 
sacred  documents  than  as  practical,  workable  guidelines. 


hat's  wrong  with  military  specifications 
and  standards  ? Why  have  these  documents  been 
criticized  over  the  years  by  the  defense  indus- 
try, program  managers,  and  military  users? 
Both  Congress  and  the  General  Accounting 
Office  have  condemned  specifications  and  stand- 
ards as  sources  of  poor  performance,  goldplat- 
ing, excessive  delays,  and  unnecessary  costs. 
But  is  the  real  problem  the  documents  and  the 
way  they  are  written,  or  is  it  the  way  they  are 
used? 

All  responsible  organizations,  whether  in- 
volved in  commercial  or  government  operations, 
must  utilize  specifications  to  some  degree.  To- 
gether with  standards,  they  represent  a readily 
available  library  of  accepted  design,  assembly, 
test,  inspection,  and  management  techniques 
which  are  known  consistently  to  produce  quality 
products.  When  properly  prepared,  they  prevent 
“reinvention  of  the  wheel.”  Specifications  and 
standards  are,  by  far,  the  most  effective  means 
available  for  communicating  requirements  be- 


tween the  customer  and  the  seller,  and  as  such 
are  essential  procurement  instruments.  Why, 
then,  have  they  been  so  maligned? 

DSB  Task  Force 

In  July  1974,  at  the  request  of  the  Deputy 
Secretary  of  Defense,  the  Defense  Science 
Board  created  a task  force  on  specifications  and 
standards  to  answer  these  specific  questions. 
With  representation  from  the  Office  of  the 
Secretary  of  Defense,  the  three  military  depart- 
ments, the  Defense  Logistics  Agency,  and  in- 
dustry, the  task  force  had  as  its  primary  objec- 
tive the  development  of  recommendations  for 
improving  the  origination,  generation,  mainte- 
nance, and  application  of  specifications  and 
standards. 

The  members  began  by  evaluating  the  con- 
clusions, recommendations,  and  implementation 
of  recommendations  developed  from  previous 
acquisition  and  standardization  program 


by  Lester  Fox  Jeffrey  S.  Allan 

Director  Staff  Engineer 

Defense  Materiel  Specifications  and  Standards  Office 
OASD  (MRA&L) 
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studies.^  Unreasonable  specification  require- 
ments cited  in  many  of  these  studies  were  ex- 
amined and  then  discussed  with  government  and 
industry  managers.  The  task  force  evaluated  a 
sampling  of  current  Department  of  Defense 
specifications  and  standards,  related  DoD  or- 
ganizational structure,  application  practices, 
and  feedback  mechanisms,  and  reached  the  fol- 
lowing conclusions : 

• Specifications  and  standards  are  essential 
tools  in  the  DoD  procurement  process. 

• Although  not  perfect,  the  existing  body  of 
military  specifications  and  standards  adequately 
satisfies  the  needs  of  DoD. 

• The  climate  surrounding  the  application, 
interpretation,  and  enforcement  of  specifica- 
tions and  standards  needs  immediate  improve- 
ment. 

^ V.  S.  Bureau  of  the  Budget,  “Review  of  the  DoD 
Standardization  Program,”  Washington,  DC,  December 
H,  1961;  Logistics  Management  Institute  Report,  “The 
Department  of  Defense  Standardization  Program," 
Washington,  DC,  November  1963;  Defense  Advisory 
Council,  “Recommendations  for  Specifications  Simplifi- 
cation,” January  11,  196^;  U.  S.  Congress,  House,  “Mili- 
tary Supply  Systems:  Cataloging,  Standardization,  and 
Provisioning  of  Spare  Parts,”  Congressional  Record, 
p.  91-1,718;  Assistant  Secretary  of  Defense  (I&L), 
“DoD  Standardization  Study,”  February  22, 1972. 


• Certain  categories  of  specifications  and 
standards,  either  by  their  structure  or  misuse, 
display  a tendency  to  drive  contract  costs  un- 
necessarily. These  documents  are  typically  non- 
end-item oriented  in  that  they  define  procedural 
techniques. 

The  study  found  that  although  some  improve- 
ment in  the  substantive  content  of  military  and 
federal  specifications  and  standards  was  pos- 
sible in  all  areas,  the  benefits  derived  from  such 
improvement  would  not  be  as  significant  or 
achievable  in  the  near  term  as  those  which  could 
be  achieved  from  an  improved  climate  of  appli- 
cation. Specifically,  the  task  force  determined 
that  excessive  costs  arose  when  the  specifica- 
tions and  standards  imposed  on  a contract 
required  change  by  contractors  from  their 
accustomed  and  proven  methods  of  doing  busi- 
ness ; were  given  their  most  stringent  interpre- 
tations ; were  too  rigidly  or  prematurely  applied 
in  the  acquisition  cycle;  were  input-oriented 
rather  than  objective-oriented;  were  applied 
without  limiting  incorporation  by  reference  of 
lower-tier  documents  called  out  in  the  top-level 
documents ; or  were  used  in  a climate  requiring 
excessive  demonstration  of  compliance  through 
prescribed  procedural  approaches.  Misuses  in- 
cluded misapplications,  overapplication,  under- 
application, untimely  application,  and  misinter- 
pretation. 

Unnecessary  Requirements 

Although  the  structure  and  layout  of  specifi- 
cations and  standards  sometimes  contributed 
to  their  misapplication,  investigation  revealed 
that  the  flexibility  and  levels  of  applicability 
written  into  specifications  and  standards  were 
consistently  ignored,  resulting  in  inappropriate, 
excessive,  and  costly  contract  requirements. 
Specifications  and  standards  were  treated  as  if 
they  were  sacred.  Contractors  and  government 
managers  were  found  to  be  equally  at  fault,  but 
for  different  reasons. 

Government  managers  responsible  for  gen- 
erating requirements  took  great  pains  to  avoid 
the  risk  of  failure,  often  going  to  extremes  to 
preclude  that  risk ; while  contractors  were  con- 
strained by  the  pressures  of  a highly  competi- 
tive marketplace  to  comply  indiscriminately 
with  requirements  even  when  those  require- 
ments were  obsolete,  redundant,  or  not  cost- 
effective.  In  most  cases,  little  analysis  had  been 
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done  to  evaluate  the  costs  of  certain  specifica- 
tions as  opposed  to  their  benefits.  The  result  was 
a tendency  toward  an  overly  conservative  and 
unnecessarily  expensive  application  of  military 
specifications  and  standards. 

Of  the  more  than  40,000  documents  cata- 
logued in  the  DoD  Index  of  Specifications  and 
Standards  covering  both  hardware  and  man- 
agement subjects,  the  task  force  determined 
that  nonhardware  specifications  and  standards 
had  the  greatest  potential  for  misuse.  Typically, 
in  addition  to  improving  the  potential  for  qual- 
ity performance,  specifications  serve  as  a 
primer  of  good  management  practices  for  the 
inexperienced.  Many  of  the  military’s  manage- 
ment standardization  documents,  however,  re- 
quired contractors  to  develop  a set  of  proce- 
dures (for  quality  control,  reliability,  or  config- 
uration management,  for  example)  different 
from  acceptable  procedures  already  established 
within  their  organization. 


'^Specifications  are,  by  far,  the  most 
effective  means  available  for  com- 
municating requirements  between 
the  customer  and  the  seller,  and  as 
such  are  essential  procurement  in- 
struments." 


Based  on  these  findings,  the  task  force  rec- 
ommended a twofold  approach  to  improve  the 
climate  for  applying  specifications  and  stand- 
ards. First,  it  advocated  increased  emphasis  on 
tailoring  requirements  to  specific  system  needs 
prior  to  contractual  application.  Specifications 
and  standards  reviews,  for  example,  should  be 
conducted  by  the  government  prior  to  issuing 
the  request  for  proposals.  During  these  reviews, 
attention  should  be  focused  on  ensuring  not  only 
that  all  facets  of  the  program  are  adequately 


addressed,  but  also  that  only  the  most  appro- 
priate documents  are  selected  and  only  their 
essential  requirements  invoked.  Contractors 
should  be  encouraged  to  propose  cost-effective 
alternatives  to  government  requirements.  In 
concert  with  the  design-to-cost  philosophy, 
contractor  flexibility  should  be  promoted  and 
encouraged. 


"Although  not  perfect,  the  existing 
body  of  military  specifications  and 
standards  adequately  satisfies  the 
needs  of  DoD." 


The  second  recommendation  was  to  strength- 
en top-level  DoD  management  of  specifications 
and  standards.  The  task  force  concluded  that 
closer  management  attention  should  be  aimed 
at  controlling  the  format,  content,  and  prolifer- 
ation of  those  specifications  and  standards  that 
had  the  greatest  potential  for  misuse.  Improved 
feedback  mechanisms  should  be  developed  to 
couple  specification  preparers  with  government 
and  industry  specification  users.  New  specifica- 
tions and  standards  should  be  structured  to 
facilitate  their  tailoring. 

DoD  Actions 

As  a result  of  the  task  force’s  findings,  the 
Deputy  Secretary  of  Defense  instructed  the  De- 
fense Materiel  Specifications  and  Standards 
Office  and  the  military  departments  to  review 
and  evaluate  the  process  of  establishing  tech- 
nical requirements  for  inclusion  in  RFPs  and 
contracts.-  Specific  emphasis  was  to  be  placed 
on  assuring  coordination  and  interaction  among 
the  contributing  technologies  in  those  areas 
known  to  have  a high  potential  for  generating 
costs.  In  addition,  the  Deputy  Secretary  di- 
rected that  these  organizations  should : 

Institute  'procedures  and  policies  to  con- 
trol  blanket  contractual  imposition  of  such 
specifications  and  standards.  These  con- 
trols should  he  structured  to  force  tech- 
nical activities  to  tailor  requirements  to 
the  essential,  specific,  operational  needs  of 
the  end  item  equipment  or  system.^ 

Meanwhile  the  military  departments  took 

• Deputy  Secretary  of  Defense  Memorandum,  August 
h,  1975,  subject:  Specifications/ Standards  Application. 
3 Ibid. 
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steps  to  improve  the  application  and  tailoring 
of  specifications  and  standards.  The  U.  S.  Army 
Development  and  Readiness  Command,  the 
Chief  of  Naval  Material,  and  the  Air  Force 
Systems  Command  established  procedures  and 
issued  instructions  to  their  subordinate  com- 
mands for  conducting,  reviewing,  and  reporting 
tailoring  activity.*-®’®  Concurrently,  the  Armed 
Services  Procurement  Regulations  were  revised 
to  mandate  the  tailoring  of  specifications  and 
standards  (ASPR  1-1201  (a))  and  the  feedback 
of  contractual  changes  affecting  specifications 
and  standards  (ASPR  1-1201  (e)). 


''The  concept  of  tailoring  contractual 
requirements  to  satisfy  system  re- 
quirements is  not  new.  It  is  basic, 
good  management." 


DMSSO  also  initiated  a series  of  briefings 
with  heads  of  the  various  system  commands  to 
review  their  methods  of  implementing  specifi- 
cations and  standards  application  policies.  A 
program  of  on-site  visits  and  discussions  was 
initiated  with  the  DoD  components  and  pro- 
gram managers  to  determine  the  degree  to 
which  the  services  were  implementing  the  new 
policies  and  to  view  first-hand  the  difficulties 
that  were  encountered. 

The  information  gained  from  these  visits  and 
other  related  activities  made  it  evident  that 
policies  governing  the  application  and  tailoring 
of  specifications  and  standards  should  be  more 
clearly  defined  and  specific  responsibilities 
assigned  to  DoD  components  and  program 
managers.  Consequently,  on  April  9,  1977,  DoD 
Directive  4120.21,  “Specifications  and  Standards 
Application,”  was  issued.  This  directive  requires 
all  DoD  components  to  establish  specific,  con- 
tinuing management  controls  over  the  utiliza- 
tion of  specifications,  standards,  and  related 
technical  data  in  the  acquisition  process  to 
assure  that  they  are  properly  applied  and 

* U.  S.  Army  Materiel  Command  Letter,  September 
25,  1975,  subject:  Elimination  of  Nonessential  Contract- 
ual Requirements  of  Specifications  and  Standards  in 
RFPs  and  Subsequent  Contracts. 

5 U.  S.  Naval  Material  Command  Letter,  October  7, 
1975,  subject:  Specifications  and  Standards  Application. 

® U.  S.  Air  Force  Systems  Command,  AFSC  Regula- 
tion 800-25,  “Application  of  Military  Specifications  and 
Standards  to  DoD  Procurements,”  June  12,  1975. 


tailored  to  reflect  the  minimal,  essential  re- 
quirements for  the  particular  system.  In  addi- 
tion, the  new  directive  specifies  that  existing 
management  review  boards  assure  that  tailoring 
is  accomplished,  that  records  are  maintained 
as  to  the  degree  of  tailoring  accomplished,  and 
that  feedback  is  requested  from  potential 
contractors  during  the  solicitation  stage. 

Tailoring  Philosophy 

The  concept  of  tailoring  contractual  require- 
ments to  satisfy  system  requirements  is  not 
new.  It  is  basic,  good  management.  At  the 
outset  of  any  new  defense  system  development 
program,  it  is  not  practicable  to  define  and 
describe  all  technical  requirements  down  to  the 
last  detail  which  ultimately  will  be  required  for 
production  of  the  system.  The  development  of 
the  definitive  detail  into  the  system  specification 
is  a progressive,  evolutionary  process  which 
continues  throughout  the  development  and  into 
the  production  phases  of  the  program.  At  every 
major  milestone  in  program  development,  re- 
views are  conducted  to  ensure  that  all  aspects 
of  system  requirements  have  been  or  will  be 
adequately  addressed.  All  military  specifications 
and  standards  which  enhance  the  success  of  the 
program  are  incorporated  into  the  system  spe- 
cification and  associated  statement  of  work. 


"Certain  categories  of  specifications 
and  standards,  either  by  their  struc- 
ture or  misuse,  display  a tendency 
to  drive  contract  costs  unnecessarily. 
These  are  typically  non-end-item  ori- 
ented in  that  they  define  procedural 
techniques." 


In  the  past,  this  evolutionary  process  focused 
on  military  specifications  and  standards  as  a 
whole  unless  major  deficiencies  were  found 
within  required  standardization  documents.  The 
detailed  challenge  of  individual  requirements  to 
detect  redundant,  unnecessary,  or  untimely  de- 
mands contained  within  essential  specifications 
is  typically  not  addressed. 

The  current  DoD  emphasis  on  specifications 
and  standards  not  only  expands  upon  previous 
practices  but  focuses  on  specific,  formalized 
tailoring  and  application.  This  involves  four 
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basic  steps  (Figure  1).  A degree  of  caution 
must  be  exercised  in  achieving  the  desired  bal- 
ance between  performance  and  life  cycle  cost. 
While  it  is  essential  that  specifications  and 
standards  be  applied  in  a prudent  and  cost- 
effective  manner,  essential  operational  capa- 
bility requirements  must  not  be  sacrificed. 

The  process  described  is  particularly  appli- 
cable in  the  selection  and  use  of  nonproduct 
specifications  and  standards  in  a given  materiel 
acquisition  program.  On  the  other  hand,  prod- 
uct specifications  such  as  those  for  parts,  mate- 
rial, components,  and  equipments  are  not  as 
susceptible  to  tailoring  when  selected  for  use 
in  a materiel  acquisition,  since  they  have  been 
developed  around  a specific  set  of  design  or 
performance  characteristics.  The  process  of 
tailoring  the  requirements  contained  in  the 
product  specification  would  normally  be  accom- 
plished during  the  development  and  promulga- 
tion of  the  document,  independent  of  a specific 
acquisition  program.  For  that  reason,  the  deci- 
sion regarding  the  use  of  product  specifications 
in  materiel  acquisitions  is  normally  a matter 
of  adopting  it  in  full  or  rejecting  it  completely. 
This  does  not  mean  that  product  specifications 
cannot  be  tailored  for  use  in  a particular  acqui- 
sition program ; they  can  and  should  be  when 
conditions  warrant  such  action. 

Improving  Control 

The  investigation  conducted  by  the  Defense 
Science  Board  task  force  highlighted  the  need 
for  closer  management  attention  on  controlling 
the  format,  content,  proliferation,  and  appli- 
cation of  nonproduct  specifications  and  stand- 
ards. The  defense  standardization  program  is 
commodity-oriented  and  defined  by  federal  sup- 
ply classes.  The  noncommodity-oriented  stand- 


ardization documents  (those  not  fitting  within 
specifics  FSCs)  had  been  somewhat  neglected, 
resulting  in  overlapping,  repetitive  documenta- 
tion and  voids.  To  correct  this  problem,  the 
task  force  recommended  that  a comprehensive, 
top-level  management  program  be  initiated 
which  would  identify  specific  management 
standardization  areas ; integrate  all  ongoing 
projects  in  these  areas  together  with  their  ob- 
jectives, schedules,  and  required  resources;  and 
define  future  necessary  projects  and  require- 
ments for  improving  documentation  control. 

The  task  force  developed  an  initial  listing  of 
nine  DoD  standardization  areas  needing  im- 
mediate attention  (Figure  2). 


'The  current  DoD  emphasis  on  speci- 
fications and  standards  not  only  ex- 
pands upon  previous  practices  but 
focuses  on  specific,  formalized  tailor- 
ing and  application." 


In  December  1976,  the  Defense  Materiel 
Specifications  and  Standards  Office  assigned  to 
various  lead-service  activities  the  centralized 
management  responsibility  for  developing  DoD 
program  plans  to  accomplish  interservice  stand- 
ardization of  appropriate  documents.  For  each 
of  the  assigned  areas,  the  lead-service  activities 
are  required  to  prepare,  coordinate  and  provide 
DMSSO  with  a five-year  standardization  docu- 
mentation program  plan  to : 

• Identify  existing  military  documents  and 
related  data  item  descriptions. 


Figure  1.  Basic  Steps  in 
Specifications  Application 

• Selecting  those  documents  that  may  have  appli- 
cation to  a particular  acquisition  program. 

• Identifying  from  these  the  ones  that  have  specific 
application. 

• Tailoring  each  document  to  include  only  the 
minimum  necessary  requirements  in  the  solici- 
tation or  contract. 

• Specifically  tailoring  these  requirements  to 
support  the  particular  system  during  acquisition 
and  life  cycle  ownership. 


14 


October  1977 


• Assess  the  status  of  these  documents  with 
respect  to  current  military  requirements  and 
the  state-of-the-art. 

• Highlight  voids  in  document  requirements. 

• Plan  for  deleting  unnecessary  require- 
ments, eliminating  duplication,  and  consoli- 
dating specifications,  standards,  and  related 
documents. 

• Restructure  and  develop  specifications  and 
standards  for  cost-effective  application. 

DMSSO  also  expanded  the  list  of  areas  need- 
ing improved  management  attention.  These  in- 
cluded such  nonproduct-oriented  disciplines  as 
nondestructive  testing  and  inspection,  thermal 
joining  of  metals,  soldering,  electromagnetic 
compatibility,  and  long-haul  communications. 
A reliability  standardization  document  pro- 
gram plan  was  approved  by  the  Defense  Ma- 
teriel Specifications  and  Standards  Board,  and 
draft  program  plans  were  developed  for  nearly 
all  of  the  remaining  areas. 


"'While  it  is  essential  that  specifica- 
tions and  standards  be  applied  in 
a prudent  and  cost-effective  manner, 
essential  operational  capability  re- 
quirements must  not  be  sacrificed." 


In  conjunction  with  the  emphasis  placed  on 
restructuring  specifications  .and  standards  to 
make  them  more  conducive  to  selective  applica- 
tion and  tailoring,  the  concept  of  sectionaliza- 
tion  was  introduced.  In  simplest  terms,  this 
promotes  the  construction  of  standardization 


Figure  2.  Standardization  Areas 
Requiring  Management  Emphasis 

• General  design  requirement  specifications. 

• Environmental  requirements  and  test  methods. 

• Reliability  and  maintainability. 

• Quality  control. 

• Human  factors  and  safety. 

• Documentation. 

• Configuration  control. 

• Integrated  logistic  support. 

• Packing,  packaging,  preservation,  and  trans- 
portation. 


documents  to  simplify  the  tailoring  process  by 
specifically  grouping  all  mandatory  require- 
ments, specifically  identifying  optional  require- 
ments, ranges,  variables,  and  the  like,  and  struc- 
turing each  requirement  to  be  independent  of 
any  other  requirement  in  the  document.  The 
purpose  and  objective  of  each  separately  struc- 
tured requirement  are  defined  together  with  a 
statement  of  how  it  should  be  utilized  in  acqui- 
sition programs.  This  sectionalization  approach 
has  been  utilized  successfully  on  such  documents 
as  MIL-STD-202,  “Test  Methods  for  Electronic 
and  Electrical  Component  Parts,”  and  MIL- 
STD-810,  “Environmental  Test  Methods,”  and 
the  expansion  of  the  concept  and  its  application 
to  other  standardization  areas  is  being  fostered. 

Program  Assessment 

Results  from  major  defense  system  reviews 
conducted  by  the  DMSSO  have  been  encourag- 
ing. Because  of  the  DoD  emphasis  on  specifica- 
tion tailoring  and  selective  application,  fiexibil- 
ity  provisions  were  incorporated  into  the  solici- 
tation and  contractual  packages  for  major  ac- 
quisition programs  such  as  the  Navy’s  Elec- 
tronic Warfare  Suite  and  the  Army’s  Single 
Channel  Ground  and  Airborne  Radio  Subsystem 
(SINCGARS)  and  Advanced  Attack  Helicopter 
programs. 

Competing  contractors  for  the  EW  suite  and 
the  AAH  were  provided  with  considerable  de- 
sign latitude  during  the  development  of  their 
system  specifications.  In  issuing  the  RFP  for 
the  prototype  EW  suite  development  contract, 
the  Naval  Electronics  Systems  Command  clear- 
ly specified  mandatory  operational  performance 
requirements.  Many  of  the  military  specifica- 
tions and  standards  cited  in  the  statement  of 
work,  however,  were  provided  to  the  contractors 
to  be  used  as  technical  guidelines.  In  a similar 
vein,  the  AAH  competing  contractors  were  en- 
couraged to  submit  waivers,  deletions,  and  al- 
ternatives to  specifications  and  standards  cited 
in  the  government-prepared  system  specifica- 
tion which  the  contractor  felt  were  cost-effective 
and  appropriate. 

The  SINCGARS  program  extended  the  speci- 
fications and  standards  flexibility  provisions  to 
include  contractual  latitude  in  the  submission 
of  technical  data.  Offerors  were  requested  to 
propose  alternative  data  items,  cross-referenced 
to  stated  requirements,  with  justification  and 
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cost  estimates  for  the  Army  Electronics  Com- 
mand’s evaluation. 

Progress  is  clearly  being  made  in  improving 
the  climate  of  specifications  and  standards  ap- 
plication, tailoring  and  control.  A comprehen- 
sive program  has  been  initiated  to  control  and 
coordinate  requirements  cited  in  management- 
oriented  documents.  The  gulf  between  specifica- 
tion preparers  and  specification  users  is  being 
narrowed.  Flexibility,  judgment,  and  contract- 
ual latitude  have  been  introduced  in  the  appli- 
cation of  specifications,  but  there  is  still  room 
for  significant  improvement.  The  leadership  of 
managers  at  all  levels  is  needed  to  educate  and 
motivate  both  government  and  contractor  per- 
sonnel in  the  continued,  cost-effective  develop- 
ment and  application  of  specifications  and 
standards.  DMJ 
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Operational  Influences 
on  Avionics  ReliabiUty 


Despite  the  much-publicized  horror  stories  about  avionics 
equipment  unreliability,  the  average  level  of  the  field  operational 
MTBF  achieved  for  Air  Force  avionics  equipment  is  51  percent 
of  that  required  by  co?itract  when  compared  by  the  same  failure 
definitions  and  time  base. 


fjxperience  has  shown  that  the  reliability 
performance  of  U.  S.  Air  Force  avionics  equip- 
ments in  the  field  does  not  generally  agree  with 
the  predicted  or  demonstrated  values.  In  recent 
years  logistic  support  costs  of  such  equipments 
have  become  increasingly  prohibitive ; hence,  an 
increased  interest  in  identifying  the  causes  of 
the  differences,  as  well  as  a requirement  for 
better  predictive  methods  for  estimating  field 
operational  reliability.^-^-®-^ 

An  underlying  principle  connecting  the  re- 
quired, predicted,  and  demonstrated  mean  time 
between  failure  of  avionics  equipment  is  that 
each  index  attempts  to  measure  the  same 
parameter;  namely,  the  inherent  reliability  of 
the  equipment.  Certainly  some  disparity  among 
these  indices  is  expected  due  to  the  different 
procedures  that  are  employed  to  generate  them. 

The  required  MTBF  is  established  by  con- 
sideration of  mission  requirements,  costs,  pre- 
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vious  experience,  and  so  forth.  The  predicted 
MTBF  is  an  analytical  assessment  of  the  in- 
herent reliability  based  on  equipment  design 
characteristics,  while  the  demonstrated  MTBF 
assesses  inherent  reliability  on  the  basis  of  the 
failure  and  operating-time  experience  of  a 
specific  sample  of  equipment  under  controlled 
laboratory  test  conditions.  The  field  MTBF,  on 
the  other  hand,  is  an  assessment  of  the  achieved 
operational  reliability  of  the  equipment  in 
actual  operation. 

In  principle,  therefore,  the  four  indices  should 
be  reasonably  alike  since  they  are  measures  of 
the  same  parameter.  In  practice,  however,  it 

1.  S.  Miller  et  al.,  “Research  Study  of  Radar  Relia- 
bility and  Its  Impact  on  Life  Cycle  Costs  for  the  APQ 
113,  -lilt,  -120,  and  -lUU  Radars,"  ASD-TR-73-22, 
Aeronautical  Systems  Division,  Wright-Patterson  Air 
Force  Base,  OH,  April  1973. 

2.  B.  Swett,  “Avionics  Reliability  Study-Phase  II," 
Air  Force  Systems  Command,  Andrews  Air  Force  Base, 
MD,  March  197U. 

3.  A.  Burkhard  et  al.,  “Environmental  Effects  on 
Reliability  and  Maintainability  of  Air  Force  Avionics 
Equipment,"  AFFDL-TR-7i-113,  Air  Force  Flight  Dy- 
namics Laboratory,  Wright-Patterson  Air  Force  Base, 
OH,  August  1974-. 

h.  J.  Nelson  et  al.,  “A  Weapon-System  Life-Cycle 
Overview : The  A—7D  Experience,"  Headquarters  Unit- 
ed States  Air  Force,  Washington,  DC,  October  197i. 
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has  been  reported  that  the  ratio  of  equipment’s 
demonstrated  MTBF  to  its  field  MTBF  can 
range  from  7 to  1 to  20  to  1.®  Even  greater 
disparities  are  quoted  when  comparing  pre- 
dicted MTBFs  to  field  MTBFs. 

In  view  of  the  foregoing,  a study  of  opera- 
tional influences  on  reliability  (OIR)  of  avionics 
equipment  was  sponsored  by  the  Air  Force 
Systems  Command.  The  OIR  study  examined 
a broad  cross  section  of  Air  Force  avionics 
equipments  in  order  to  assess  the  differences 
in  MTBFs  and  to  identify  the  significant  opera- 
tional factors  which  influence  the  reported 
disparity.®-^  In  general  the  view  taken  is  that 
any  reasonable  disparity  may  be  attributed  in 
the  main  to  either  definitional  factors,  opera- 
tional factors,  or  environmental  factors.®*®  This 
study  has  shown  that  after  removal  of  the 
definitional  factors,  about  half  of  the  remaining 
differences  are  accounted  for  by  operational 
factors  and  the  remainder  are  due  to  the  com- 
bined influence  of  environmental  and  other  fac- 
tors. 

The  scope  of  the  OIR  study  covered  the  first 
two  factors  and  included  16  different  avionics 
equipment  items  in  current  operational  use  on 
10  different  aircraft  weapon  systems.  Since 
many  of  these  equipments  were  used  on  two  or 
more  different  weapon  systems,  a total  of  30 
different  applications  (combinations  of  equip- 


5.  A.  Burkhard  et  al.,  “Environmental  Effects." 

6.  The  work  described  in  this  paper  was  sponsored 
by  the  Air  Force  Systems  Command,  Rome  Air  Devel- 
opment Center,  Griffiss  Air  Force  Base,  NY.  The  au- 
thors are  grateful  to  Mr.  Eugene  Fiorentino,  the  tech- 
nical monitor,  for  his  constructive  guidance  during  the 
study. 

7.  George  Kern  and  T.  Dmas,  “Operational  Infho- 
ence  on  Reliability,”  RADC-TR-7 6-366,  ADA  A035016, 
Rome  Air  Development  Center,  Griffiss  Air  Force  Base, 
NY,  December  1976. 

8.  A separately  conducted  study  by  Grumman  Aero- 
space Corporation  investigated  the  area  of  environmen- 
tal differences  between  required,  predicted,  demon- 
strated, and  field  operational  reliability  of  airborne 
avionics  equipment.  The  results  of  the  Grumman  study 
are  based  on  avionics  equipments  in  current  operational 
use  on  four  different  types  of  naval  aircraft.  Except 
for  the  obvious  differences  in  carrier-based  operations, 
it  is  believed  that  all  other  environments  are  relatively 
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9.  G.  Hirschberger  et  al.,  “Evaluation  of  Environ- 
mental Profiles  for  Reliability  Demonstration,"  RADC- 
TR-75-2i2,  Rome  Air  Development  Center,  Griffiss  Air 
Force  Base,  NY,  September  1975. 


ment  and  weapon  systems)  were  included  in  the 
study.  The  resultant  avionics  population  pro- 
vided a good  representation  of  the  four  major 
avionics  functions  and  an  opportunity  to  ob- 
serve operational  performance  of  the  same 
equipment  on  different  types  of  aircraft  or  in 
different  mission  environments. 

MTBF  Assessment 

A good  deal  of  confusion  has  been  generated 
by  inconsistencies  in  the  criteria  used  for  assess- 
ing field  operational  MTBF.  These  inconsisten- 
cies result  from  differences  in  definitions  used, 
for  example,  for  such  things  as  time  or  failure. 
In  the  Air  Force  Logistics  Command’s  product 
performance  assessments,  weapon  system  flying 
hours,  rather  than  equipment  operating  hours, 
are  used  as  the  time  base.^®  Hence,  equipment 
operating  time  accrued  during  ground  operation 
or  during  operational  checkouts  and  mainte- 
nance is  not  included.  This  is  in  conflict  with 
the  operating-hours  time  base  used  for  require- 
ments, prediction,  or  demonstration  test  assess- 
ments. Similarly,  the  criteria  used  by  AFLC 
for  classifying  maintenance  actions  as  failures 
include  many  items  which  would  not  be  counted 
as  relevant  for  reliability  assessment  purposes. 
The  reason  for  these  differences  in  failure  cri- 
teria stems  from  the  fact  that  the  AFLC  assess- 
ments are  logistics  resource  impact-oriented 
(logistics  MTBF)  and  not  inherent  reliability 
(mission  success) -oriented.  In  the  latter  case, 
externally  caused  failures  (broken,  bent,  loose, 
missing,  cracked,  and  so  forth)  are  normally 
counted  as  nonrelevant  failures  for  inherent 
reliability  assessment,  but  as  failures  for  AFLC 
assessment  purposes,  since  they  impact  on 
logistic  resources. 

Field  operational  reliability  performance  as- 
sessments were  based  on  analysis  of  Air  Force 
maintenance  data  using  the  standard  AFLC 
product  performance  assessment  methods,  ad- 
justed as  appropriate  to  ensure  consistency  in 
the  time  base  and  failure  criteria  used  for 
assessment. 

The  composite  effects  of  the  differences  in 
criteria  used  for  field  reliability  assessment  are 
shown  in  Figure  1.  The  graph  illustrates,  for 
each  of  the  16  different  avionics  equipment 
items  studied,  the  field  MTBF  ratios  achieved  as 

10.  Air  Force  Logistics  Command  Manual  66-15, 
“Product  Performance,”  Wright-Patterson  Air  Force 
Base,  OH,  February  1970. 
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reported  using  the  standard  AFLC  criteria,  and 
the  adjustment  to  the  MTBF  ratios  obtained 
after  reassessment  using  a consistent  criteria 
for  time  and  failure  definition. 

The  shaded  area  (logistic  MTBF  ratio)  rep- 
resents the  AFLC-reported  level  of  achieved 
field  MTBF,  whereas  the  white  area  represents 
the  adjustment  to  the  achieved  MTBF  value 
after  reassessment  of  the  data  using  a consistent 
criteria  to  derive  the  inherent  MTBF  values. 
On  the  average,  when  the  data  were  reassessed, 
the  composite  effect  of  the  adjustments  in- 
creased the  reported  MTBF  values  by  a factor 
varying  from  1.3  to  6.4  depending  upon  the 
equipment,  or  on  the  average  by  a factor  of 
2.4.  The  wide  range  of  these  definitional  ad- 
justments suggests  that  careful  attention  to 
the  criteria  used  for  reliability  assessments  is 
imperative,  if  meaningful  results  are  to  be  ob- 
tained. These  adjustments,  referred  to  as 
definitional  adjustments,  were  the  largest  single 
factor  contributing  to  the  reported  differences 
between  the  predicted,  demonstrated,  re- 
quired, and  field  operational  reliability  values. 

Operational  Influences 

The  operational  influences  on  avionics  equip- 
ment reliability  can  be  divided  into  two  cate- 
gories. The  first  comprises  maintenance  handl- 


ing factors  related  to  the  frequency  of  mainte- 
nance actions,  the  frequency  of  removal  of  the 
equipment  from  the  weapon  system  during 
maintenance,  the  shop  repair  rate,  and  so  forth. 
The  second  category,  use  factors,  consists  of 
such  elements  as  mission  duration,  weapon  sys- 
tem utilization  rate,  and  operating  to  nonoper- 
ating time  relationships.  While  the  factors 
identified  may  not  surprise  those  experienced 
in  avionics  reliability,  it  is  significant  to  note 
that  this  is  the  first  time  that  these  relationships 
have  been  comprehensively  studied,  identified, 
and  quantified. 

Implementation  of  the  regression  analysis 
procedures  to  examine  the  effects  of  18  field- 
related  factors  in  all  combinations  produced  a 
number  of  significant  equations  depicting  the 
influence  of  the  operational  factors.  The  equa- 
tions were  ordered  with  respect  to  the  demon- 
strated, predicted,  and  required  MTBF.  These 
relationships  were  developed  subsequent  to  the 
removal  of  the  definitional  factors.  In  each 
group  the  significant  measurable  factors  were 
segregated  into  two  categories : maintenance 
handling  and  equipment  use. 

Maintenance  Handling  Influence 

As  might  be  expected,  the  more  a given 
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avionics  equipment  is  subjected  to  maintenance 
handling,  the  lower  will  be  the  relative  level  of 
MTBF  achieved.  The  analysis  results  showed 
that  the  removal  rate  and  percentage  of  shop 
repairs  are  significant  measures  of  maintenance 
handling  infiuence.  These  factors  have  a strong 
negative  association  with  the  expected  field 
MTBF.  Figure  2 illustrates  the  relationship 
between  the  ratio  of  expected  field  MTBF  to 
the  demonstrated  MTBF  and  the  level  of  main- 
tenance handling  to  which  it  is  subjected.  For 
an  item  which  had  a demonstrated  MTBF  of 
400  hours,  the  range  of  expected  field  MTBF 
achieved  would  be  between  0.33  and  0.67  times 
the  demonstrated  value.  The  curves  represent 
realistic  estimates  within  which  current  gener- 
ation avionics  equipments  are  expected  to  fall. 

The  effect  of  different  maintenance  handling 
levels  on  the  ratios  is  appreciable.  Although  the 
tails  of  fitted  curves  must  be  viewed  with  some 
caution,  it  appears  that  the  larger  ratios  can  be 
associated  with  the  more  complex  hardware 
(lesser  demonstrated  MTBFs)  and  that  a higher 
level  of  maintenance  handling  tends  to  limit  the 
ratio  on  the  order  of  0.30  in  this  case. 

The  significant  use  factors  found  to  have  an 
influence  on  field  reliability  were  mission  dura- 
tion (MD),  utilization  rate  (UR),  and  the  op- 
erating to  nonoperating  time  ratio  (OPTR). 
Figure  3 shows  the  field  MTBF  that  may  be 
expected  for  a given  value  of  demonstrated 
MTBF  when  different  levels  of  equipment  use 
are  considered.  Again,  the  low,  medium,  and 
high  levels  of  use  are  assessed  by  the  clear 
separation  of  the  lines,  with  the  high  equip- 
ment use  relationship  standing  apart  from  the 
other  two. 

Following  the  same  pattern.  Figure  4 shows 
the  field  MTBF  that  may  be  expected  for  a 
given,  demonstrated  MTBF  when  the  combined 
influence  of  maintenance  handling  and  equip- 
ment use  are  taken  into  account.  Note  that 
these  curves  represent  the  range  within  which 
the  expected  field  MTBF  performance  will  fall 
using  typical  values,  not  extremes,  of  mainte- 
nance handling  and  use  factors. 

Additional  effects  of  the  combined  operational 
factors  are  assessed  in  Figure  5 on  p.  22.  In 
this  case  the  typical  factor  values  employed  are 
those  partitioned  by  aircraft  type.  It  is  ap- 
parent that  the  field  MTBF  achieved  by  avion- 
ics equipments  installed  on  heavy  bombers. 


transports,  or  tankers  is  about  twice  that 
achieved  when  the  same  equipments  are  in- 
stalled on  fighters,  interceptors,  or  trainer  air- 
craft. These  curves  indicate  that  for  purposes 
of  future  procurements  involving  either  life 
cycle  cost  or  reliability  improvement  warranty 
contracts,  some  recognition  of  the  type  of  air- 
craft the  equipment  is  to  be  installed  in  should 
be  included.  Similarly,  for  reliability  predic- 
tions, a modifier  to  the  failure  rates  used  for 
aircraft  environments  should  be  developed  to 
reflect  the  differences  observed. 

Influence  Assessed 

The  overall  influence  of  the  operational  fac- 
tors were  assessed  on  the  basis  of  the  percent 
of  variation  in  the  dependent  variable  (field 
MTBF)  that  is  explained  by  the  independent 
variables  (operational  factors)  after  accounting 
for  the  normalizing  MTBF.  On  this  basis  a 
reasonable  estimate  of  the  overall  operational 
influence  on  the  field  MTBF  relative  to  the  dem- 
onstrated MTBF  is  51  percent.  Extending  this 
line  of  reasoning,  the  remaining  unexplained 
variation,  49  percent,  would  be  a reasonable 
estimate  of  the  influence  of  other  factors  such 
as  the  different  environmental  levels  applied  to 
equipment  in  the  field.  Typically,  these  environ- 
ments would  include  vibration,  shock,  tempera- 
ture, humidity,  and  so  forth.  The  estimated  total 
operational  influence  on  the  field  MTBF  relative 
to  the  predicted  MTBF,  calculated  in  the  same 
manner,  is  45  percent. 

Figure  6 on  p.  22  provides  a composite  illus- 
tration of  the  impact  of  the  definitional  and 
operational  factors.  In  column  1,  the  ratio  of 
the  demonstrated  MTBF  to  the  field  MTBF 
based  on  raw  data  is  approximately  6 to  1, 
whereas  after  the  removal  of  the  effects  of 
the  definitional  factors,  the  same  ratio  is  2.4 
to  1.  When  the  definitional  factors  have  been 
removed,  51  percent  of  the  variation  in  the 
field  MTBF  may  be  attributed  to  the  opera- 
tional factors  and  49  percent  to  environmental 
and  other  factors.  Similarly,  column  2 shows 
the  composite  impact  relative  to  the  predicted 
MTBF. 

Performance  Assessments 

This  study  has  shown  that  despite  the  much- 
publicized  horror  stories  about  avionics  equip- 
ment unreliability,  the  average  level  of  the  field 
operational  MTBF  achieved  for  Air  Force 
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avionics  equipment  when  compared  by  the  same 
failure  definitions  and  time  base  is  51  percent 
of  the  contractually  required  MTBF,  and  49 
percent  in  relation  to  the  laboratory-demon- 
strated MTBF.  The  values  are  based  on  a 
carefully  selected  matrix  of  equipments  and 
aircraft  types,  and  represent  total  inventory 
performance  of  these  equipments  and  weapon 
systems  in  the  continental  United  States  for  a 
one-year  period. 

In  addition  to  the  quantitative  results  ob- 
tained, several  interesting  observations  of  the 
degree  to  which  avionics  field  reliability  per- 
formance varies  were  also  made.  These  included 
comparisons  of  the  same  equipments  on  the 
same  aircraft,  of  the  same  equipments  on  the 
same  aircraft  and  mission  at  different  bases,  of 
the  same  equipments  on  different  aircraft,  and 
comparisons  by  aircraft  type. 

The  first  of  the  comparative  investigations 
dealt  with  the  degree  to  which  equipment  MTBF 
might  be  influenced  by  differences  in  the  main- 
tenance environment  at  different  bases.  For 
this  investigation,  three  items  of  avionics  equip- 
ment were  selected  on  a high  performance 
supersonic  jet  aircraft  operated  in  a very  close- 
ly regulated  and  controlled  training  role  at  nine 
different  bases.  The  results  of  this  analysis  are 
summarized  in  Figure  7 on  p.  22. 

The  data  analysis  results  are  presented  in 
terms  of  the  equipment’s  achieved  MTBF  by 
base  normalized  by  the  average  MTBF  for  all 
bases.  This  permits  direct  comparison  of  the 
MTBF  performance  at  each  base  versus  the  fleet 
average  MTBF  for  each  of  the  three  different 
equipments.  The  equipments  selected  were  a 
UHF  communications  radio,  a TACAN  naviga- 
tion set,  and  an  inertial  platform,  so  that  data 
representing  the  MTBF  performance  of  the 
three  different  avionics  functions  could  be  com- 
pared at  the  nine  different  training  bases.  In 
addition  to  the  MTBF  achieved  at  each  base  for 
the  three  different  equipment  items,  the  average 
of  the  three  values  of  relative  MTBF  achieved 
for  each  base  was  calculated.  The  object  of  the 
base  average  value  is  to  smooth  out  the  effect  of 
large  variations  on  one  item  of  equipment  and 
to  provide  an  assessment  of  the  impact  on  field- 
achieved  MTBF  due  to  differences  in  the  base- 
to-base  maintenance  environment. 

While  caution  is  recommended  against  draw- 
ing inferences  relative  to  the  causal  factors  for 


Figure  2.  influence  of  Maintenance  Handling 
on  Field  MTBF  to  Demonstrated  MTBF  Ratio 
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Figure  3.  Influence  of  Equipment  Use  on 
Field  MTBF  to  Demonstrated  MTBF  Ratio 
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Figure  4.  influence  of  Combined  Operational 
Factor  on  Field  MTBF  to  Demonstrated 
MTBF  Ratio 


Defense  Management  Journal 


21 


Figure  5.  Influence  of  Combined  Operational 
Factors  on  Field  MTBF  to  Demonstrated  MTBF 
Ratio  ~ Typical  Levels  by  Aircraft  Type 


Figure  6.  Composite  Impact  of  Definitional  Factors 
and  Operational  Factors 
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Figure  7.  MTBF  Performance  Comparison  at 
Nine  Different  Bases  - Identical  Equipment 
and  Aircraft 
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1.63 

6 
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1.13 
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1.42 
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1.09 
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1.03 
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1.04 
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80.4 

88.4 
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the  differences,  it  is  evident  from  these  data 
that  fairly  large  differences  in  field  MTBF  can 
occur  even  though  the  same  equipment  is  used 
on  the  same  type  of  aircraft  flying  a virtually 
identical  training  syllabus  at  different  bases. 
An  excellent  summary  of  how  avionics  main- 
tenance is  performed  in  the  Air  Force  op- 
erating commands,  and  some  of  the  variables 
involved,  is  given  in  a recently  published  re- 
port by  AFSC.“ 

Typical  Characteristics 

To  get  a more  detailed  insight  into  the  field 
MTBF  performance  of  equipment  used  on  sev- 
eral different  aircraft,  and  the  variations  among 
the  factors  of  interest  that  were  identified 
through  the  preceding  analyses,  tabulations 
were  made  of  the  more  significant  performance 
indices  of  two  different  equipment  types,  each 
operational  on  two  or  more  different  aircraft. 
These  equipments  were  a UHF  communication 
radio  and  a TACAN  navigation  set.  The  data 
were  arranged  in  three  groups,  namely  the 
equipment  MTBF  values,  the  maintenance 
handling  factors,  and  the  use  factors  (Figures 
8 and  9). 

An  examination  of  the  MTBF  values  for  the 
UHF  radio  discloses  that  this  equipment  had 
an  MTBF  requirement  of  450  hours,  a pre- 
dicted MTBF  of  1095  hours,  and  a demon- 
strated MTBF  of  552  hours.  Proceeding  to  the 
reassessed  field  MTBF  values,  one  can  see  the 
wide  range  of  variation  of  operational  MTBF 
for  what  would  appear  to  be  the  same  equip- 
ment, performing  the  same  function,  on  six 
different  types  of  aircraft.  Note  the  extreme 
values;  MTBF  equals  107  hours  on  weapon 
system  H,  a supersonic  interceptor;  and  917 
hours  on  weapon  system  C,  a cargo  and  trans- 
port aircraft. 

Another  significant  difference  not  apparent 
in  the  data  which  should  be  considered  is  that 
on  weapon  systems  A and  C,  the  UHF  commu- 
nications installation  is  dual,  in  that  each  air- 
craft has  two  complete  UHF  communication 
sets.  This  should  not  be  interpreted  to  mean 
that  only  one  radio  is  operational  at  any  given 
time,  since  it  is  normal  practice  to  use  both  sets, 
each  tuned  to  a different  frequency.  The  effect 

11.  P.  Owens  et  ah,  “Avionics  Maintenance  Study,” 
AFAL-TR-77-90,  Air  Force  Avionics  Laboratory, 
Wright-Patterson  Air  Force  Base,  OH,  June  1977. 
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Figure  8.  Avionics  Performance  Indices  - UHF  Communication  Radio 
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of  having  dual  UHF  radio  installations  on 
weapon  systems  A and  C is  that  the  pilots  tend 
to  be  less  critical  of  the  radio  equipment  when 
a discrepancy  occurs,  because  the  crew  can 
switch  to  the  other  radio  and  complete  the 
communications,  and  consequently  they  are  less 
likely  to  report  the  discrepancy  on  the  postflight 
debriefing  form  than  if  it  were  a single  installa- 
tion on  which  they  were  solely  dependent. 

The  maintenance-handling  factors  likewise 
reflect  significant  differences  in  some  cases  and 
reasonably  similar  performance  in  others.  These 
show  both  the  range  and  magnitude  of  the 
factors  of  interest  and  reflect  total  fleet  values 
for  a one-year  period,  thereby  eliminating  much 
of  the  variation  normally  seen  when  dealing 
with  more  limited  data  samples. 

The  first  two  factors — frequency  in  terms  of 
mean  time  between  maintenance  actions 
(MTBMA)  and  maintenance  man-hours  expend- 
ed per  equipment  flight  hour  (MI) — are  in- 
dicative of  the  maintenance  resources  expended. 
The  remaining  maintenance-handling  factors 
listed  are  expressed  as  percentages  of  the  indi- 
cated “How  Malfunctioned”  or  “Action  Taken” 
codes  relative  to  the  total  number  of  equipment 
maintenance  actions  associated  with  the  equip- 
ment item.  The  values  give  a good  indication 
of  the  range  of  variation  and  absolute  values 


that  are  typical  for  this  equipment  on  the  six 
different  aircraft  systems. 

The  last  group  of  data  in  Figure  8 gives  a 
good  indication  of  the  magnitude  and  range  of 
the  three  primary  use  factors  of  interest:  mis- 
sion duration,  utilization  rate,  and  operating  to 
nonoperating  time  ratios  for  each  of  the  six 
weapon  system  applications. 

In  summary,  the  data  indicate  that  there  is  a 
considerable  range  of  variation  in  field  MTBF 
among  the  six  different  aircraft  types,  as  well 
as  variations  in  the  associated  maintenance 
handling  and  use  factors. 

The  field  performance  indices  for  the  TACAN 
navigation  set  installed  on  weapon  systems  A 
and  D are  shown  in  Figure  9.  These  are  both 
supersonic  jet  aircraft;  weapon  system  A is  a 
medium  bomber,  while  D is  a high-performance 
training  aircraft. 

The  data  presentation  format  is  the  same  as 
that  described  previously.  Examining  the 
MTBF  values,  it  is  seen  that  this  equipment 
had  an  MTBF  requirement  of  1000  hours,  a 
predicted  MTBF  of  2900  hours,  and  a demon- 
strated MTBF  of  669  hours.  The  reassessed 
field  MTBF  shows  a value  of  129  hours  on 
weapon  system  A and  106  hours  on  weapon 
system  neither  of  which  is  significantly 
different. 


Figure  9.  Avionics  Performance  Indices  - TACAN  Navigation  Set 
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The  remaining  data  presented  in  Figure  9 
illustrate  that  contrary  to  the  slightly  lower 
indicated  value  of  field  MTBF  for  the  TACAN 
on  system  D,  the  maintenance  index  (mainte- 
nance man-hours  expended  per  equipment  flight 
hour)  is  considerably  lower  than  that  reported 
for  system  A.  The  remaining  maintenance-han- 
dling factors  are  comparable  for  both  installa- 
tions. 

A review  of  the  use  factors  indicates  that 
there  is  almost  a 3 to  1 difference  in  mission 
durations  between  the  two  applications,  and 
almost  a 2 to  1 difference  in  the  utilization  rates. 
However,  the  combination  of  a high  mission 
duration  and  low  utilization  rates  or  vice-versa 
tends  to  yield  no  significant  difference  in  the 
composite  operational  influences  for  either  ap- 
plication. 

The  relative  contribution  of  operating  and 
nonoperating  equipment  failures  was  analyti- 
cally assessed  to  calculate  the  expected  number 
of  failures  based  on  given  values  of  operating 
to  nonoperating  failure-rate  ratios  and  given 
levels  of  equipment  utilization  rates.  Since  data 
on  actual  failure-rate  ratios  were  not  available, 
ratios  of  10,  20,  40,  80,  and  160  to  1 were  as- 
sumed for  this  evaluation.  The  range  of  equip- 
ment-utilization rates  used  for  the  analysis 
was  from  15  to  100  operating  hours  per  month 
per  equipment.  These  ranges  encompass  the 
range  of  values  characteristic  of  the  equip- 
ments included  in  the  study.  This  resulted  in 
the  development  of  the  nomograph  in  Figure  10, 
from  which  one  can  estimate  the  expected  dis- 
tribution of  operating  and  nonoperating  failures 
for  a given  item  of  equipment. 

Maintenance  action  data  for  a one-year  period 
on  one  equipment  type  were  then  analyzed  and 
classified  to  derive  a count  of  the  number  of 
failures  that  occurred  during  operating  and 
nonoperating  times.  To  classify  the  failures  as 
either  operating  or  nonoperating  failures,  the 
following  rationale  was  applied  to  the  “When 
Discovered”  codes  pertaining  to  each  failure 
occurrence : If  the  code  used  was  indicative  of 
either  preflight  inspection,  special  inspection, 
quality  control  check,  depot-level  maintenance, 
or  withdrawal  from  stock,  the  failure  was  con- 
sidered to  have  occurred  during  the  nonoperat- 
ing time.  All  other  failures  were  considered  as 
having  occurred  during  the  equipment  operat- 
ing time. 


Figure  10.  Equipment  Operating  and  Nonoperating 
Failure  Contribution 

Percent  Operating  Failures 


Percent  Nonoperating  Failures 
Operating  Hours  Per  Month  Per  Equipment 


The  results  of  the  failure  data  reclassification 
described  indicate  that  40  percent  of  the  fail- 
ures were  associated  with  the  nonoperating 
period  and  the  remaining  60  percent  occurred 
during  the  operating  period.  The  values  of  utili- 
zation rates  and  operating-hour  to  flight-hour 
ratios  for  the  equipment  item  investigated  were 
38.9  hours  and  1.23  hours  per  month,  respec- 
tively. From  these  values,  47.8  is  calculated  as 
the  equipment’s  utilization  rate.  Entering  the 
data  on  the  nomograph,  it  can  be  inferred  that 
the  ratio  of  operating  to  nonoperating  field 
failure  rates  for  this  equipment  is  approxi- 
mately 20  to  1. 

In  the  absence  of  more  detailed  data,  this 
approach  can  provide  a useful  estimate  of  the 
current  operational  experience  on  a typical 
equipment  item.  The  point  made  by  this  analysis 
is  that  a significant  percentage  of  failures  can 
and  does  occur  during  nonoperating  time,  yet 
the  denominator  of  the  equation 


MTBF 


Operating  Time 
Failures 


normally  includes  all  failures  observed  without 
regard  to  the  fact  that  some  of  the  failures 
occurred  during  nonoperating  time.  This  is  an- 
other source  of  the  disparity  between  demon- 
strated MTBF  and  field  MTBF,  since  demon- 
strated MTBFs  use  only  operating  failures  and 
operating  time  for  assessment  purposes. 


Conclusions  and  Recommendations 

This  study  of  16  different  avionics  equipments 
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now  operational  on  10  different  Air  Force  air- 
craft types  has  identified  a number  of  signifi- 
cant factors  which  contribute  to  the  differences 
between  field-observed  reliability  and  demon- 
strated, predicted,  or  required  MTBF  values. 

While  many  contributing  factors  were  re- 
vealed, the  primary  causes  were  found  to  be  the 
data  base  used  for  assessment,  and,  to  a lesser 
extent,  the  operational  infiuences  of  mainte- 
nance handling  and  use.  However,  generaliza- 
tions do  not  always  fit  particular  cases.  For  this 
reason,  the  prediction  models  developed  for  esti- 
mating field  operational  MTBF  were  designed 
to  give  upper  and  lower  bounds  for  the  esti- 
mated value. 

The  definitional  differences  observed  are  at- 
tributed to  the  differences  in  the  failure  criteria 
and  time  base  used  by  the  logistic  support  com- 
munity (AFLC)  which  collects  and  analyzes  the 
data,  and  the  engineering  community  (Air 
Force  Systems  Command  and  industry)  which 
establishes  requirements,  performs  predictions, 
and  conducts  reliability  demonstration  tests. 
The  definitional  differences  are  composed  of  two 
parts,  one  related  to  the  time  base  used  for 
MTBF  assessment,  the  other  to  the  failure 
criteria  used  for  assessment  purposes. 

The  following  conclusions  and  recommenda- 
tions are  offered: 

• The  AFLC-reported  MTBFs  based  on  air- 
craft flight  hours  were  increased  by  factors 
ranging  from  1.2  to  2.7  when  the  equipment 
operating  hours  were  used  to  compute  the  field 
MTBF.  The  average  increase  for  all  16  equip- 
ments was  twice  the  AFLC-reported  field 
MTBF.  In  terms  of  operational  MTBF  achieve- 
ment relative  to  the  required  MTBF,  the  AFLC- 
reported  MTBF  increased  from  0.21  to  0.42. 
When  the  failure  classifications  were  reassessed 
and  adjusted,  the  composite  effect  of  the  fail- 
ure definitional  adjustments  increased  the  field- 
to-required  MTBF  ratio  from  0.42  to  0.51. 

• The  primary  operational  influences  are 
those  related  to  maintenance  handling  and 
equipment  use.  These  operational  influences 
collectively  account  for  about  half  of  the  re- 
maining differences  after  definitional  factors 
between  the  observed  (reassessed)  field  MTBF 
and  the  demonstrated  MTBF  are  removed,  and 
about  45  percent  of  the  remaining  differences 
when  related  to  the  predicted  MTBF. 

• The  review  of  the  failure  relevancy  criteria 


revealed  that  two  related,  but  differing,  re- 
liability characteristics  are  responsible  for  the 
differences  in  failure  classification  criteria. 
These  are  the  inherent  reliability  (engineering 
oriented)  and  the  operational  reliability  (logis- 
tics support  and  operations  oriented).  Until 
these  differences  are  clearly  recognized  and 
understood,  confusion  as  to  the  meaning  of 
MTBF  will  continue  to  exist. 

• A limited  analysis  of  maintenance  action 
data  indicates  that  approximately  60  percent 
of  maintenance  actions  are  expended  on  the 
prime  avionics  equipment  items  and  the  re- 
maining 40  percent  on  system  interfaces  and 
associated  hardware.  This  suggests  that  the 
latter  two  areas  may  be  prime  targets  for  cost 
reduction  and  reliability  improvement. 

• The  differences  in  assessed  field  operational 
MTBFs  suggest  that  it  may  not  be  valid  to  use 
a single  environmental  factor  for  aircraft  with- 
out regard  to  the  type  of  aircraft  when  making 
reliability  predictions.  Possibly  the  environ- 
mental factors  given  in  Military  Handbook 
217B  should  be  adjusted  by  an  appropriate 
modifier  to  reflect  differences  in  aircraft  type. 

• Nonoperating  failures  can  make  a signifi- 
cant contribution  to  the  assessed  field  MTBF  for 
avionics  equipment.  It  is  estimated  that  between 
20  and  60  percent  of  the  failures  recorded  dur- 
ing operational  deployment  of  avionics  equip- 
ments are  actually  nonoperating  failures.  This 
suggests  the  need  to  establish  several  separate 
measures  of  MTBF,  each  directed  at  a specific 
objective.  One  would  be  used  to  determine  in- 
herent reliability  (based  on  operating  hours), 
one  to  determine  field  operational  reliability 
(based  on  flight  hours),  and  another  to  deter- 
mine logistic  support  reliability  (based  on  cal- 
endar time;  that  is,  months  or  years).  Such  an 
approach  would  contribute  significantly  to  a 
better  understanding  of  the  meaning  of  MTBFs 
and  result  in  more  accurate  data. 


GEORGE  A.  KERN  is  a Program  Manager  at 
Hughes  Aircraft  Company,  Culver  City,  CA,  with  re- 
sponsibility for  advanced  study  programs  for  the  Design 
Effectiveness  Department  of  the  Aerospace  Groups. 

Mr.  Kern  has  some  28  years  of  industry  experience, 
with  specific  expertise  in  the  design,  development,  and 
test  and  evaluation  of  electronics  equipments  for  mili- 
tary weapon  systems. 

Mr.  Kern  has  a B.S.  degree  from  Newark  College 
of  Engineering,  Newark,  NJ,  and  has  done  graduate 
work  at  Stevens  Institute  of  Technology  and  at  UCLA. 
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Taking  a Look  at 
the  Request  for  Proposal 


This  very  basic  instrument  in  the  procurement  process  needs 
careful  consideration  by  DoD  and  industry  alike. 


w hen  the  government  pur- 
chases large,  complex  weapon 
systems,  the  request  for  proposal 
is  one  of  the  key  instruments  of 
communication  between  the  pro- 
curement officer  and  the  seller  of 
the  product  or  services.  The  po- 
tential seller  must  understand 
clearly  the  contents  of  the  re- 
quest for  proposal  in  order  to 
respond  properly  to  the  procure- 
ment requirements.  The  request 
for  proposal,  however,  has  been 
criticized  variously  for  being  too 
long,  for  requiring  too  much  non- 
essential  data  from  the  offeror, 
and  in  general  for  being  a poor 
communications  instrument. 

In  1975  a study  was  conducted 
under  the  auspices  of  the  Busi- 
ness Research  Management  Cen- 
ter, Wright-Patterson  Air  Force 
Base,  to  determine  how  contract 
administrators  viewed  the  re- 
quest for  proposal  as  the  first 
communications  instrument  in 
the  acquisition  process.  The 
study  was  designed  to  solicit  the 
opinions  of  major  defense  con- 
tractors. It  was  preceded  by  a 
review  of  the  literature  regard- 
ing the  request  for  proposal  to 
determine  what  actions  had  pre- 
viously been  taken. 

In  the  literature  search,  two 
prior  studies  were  located.  Both 
studies  were  conducted  in  1969 : 
one  was  completed  by  an  Air 
Force  team  of  procurement  spe- 
cialists; the  other,  by  members 
of  the  Aerospace  Industries  As- 
sociation of  America. 

The  studies  followed  similar 
formats.  Each  included  a critical 


by  Cecil  V.  Hynes 
Associate  Professor  of  Marketing 
University  of  Maryland 
College  Park,  MD 

Opinions  expressed  herein  are  those  of  the 
author  and  not  necessarily  those  of  the  De- 
partment of  Defense. 


evaluation  of  several  requests 
for  proposals  and  a review  of  the 
regulations  and  directives  that 
applied  to  the  procurement 
process. 

The  Air  Force  study  reviewed 
in  detail  three  requests  for  pro- 
posals which  were  typical  of  re- 
cently completed  formal  source 
selection  activities.  The  RFPs  se- 
lected were:  the  second  source 
procurement  for  the  Minuteman 
III  Guidance  and  Control  Set 
contracted  by  the  Space  and  Mis- 
sile Systems  Organization,  the 
Aeronautical  Systems  Division 
AGM-65A  Maverick  Missile  pro- 
curement, and  the  Electronic 
Systems  Division  TPN-19  Land- 
ing Control  central  procure- 
ment.^ 

The  Aerospace  Industries  As- 
sociation study  used  proposals 
for  the  short-range  attack  mis- 
sile, the  C-5A  cargo  airplane,  and 
the  airborne  warning  and  con- 
trol system  as  case  studies  to 
demonstrate  industry’s  experi- 
ence in  working  with  requests 
for  proposals.^ 

Both  studies  made  similar, 
fundamental  recommendations. 
For  example,  each  recommended 
page  limitations  for  the  requests 
for  proposals  and  for  the  con- 
tractors’ proposals.  Both  studies 
also  recommended  a careful 
screening  of  the  requirements 
placed  on  contractors  for  the  pro- 
vision of  data,  certificates,  and 
management  reports.® 

The  only  published  articles 
located  during  the  literature 
search  which  dealt  directly  with 
the  request  for  proposal  appear- 
ed in  the  January  1973  issue  of 
the  Defense  Management  Jour- 
nals These  articles  generally 
characterized  the  requests  for 
proposals  as  excessively  long 
documents,  written  in  such  a 
manner  as  to  encourage  contrac- 


tors and  responding  companies 
to  place  a large  volume  of  non- 
essential  information  in  their 
proposals.  Many  of  the  points 
raised  in  the  Defense  Manage- 
ment Journal  articles  were  the 
same  as  those  addressed  in  the 
Air  Force  and  Aerospace  Indus- 
tries Association  studies. 

In  May  1973,  the  Aeronautical 
Systems  Division  of  the  Air 
Force  Systems  Command  pub- 
lished ASD  Pamphlet  800-6  to 
provide  guidance  for  improving 
and  simplifying  the  request  for 
proposal.®  This  publication  sug- 
gested reorganizing  the  request 
for  proposal  in  a format  that 

^Assistant  Secretary  of  the  Air 
Force  (Procurement),  “Air  Force  Re- 
quest for  Proposal,  Study  Team  Final 
Report,"  Washington,  DC,  1969. 

^Aerospace  Industries  Association  of 
America,  “An  AIA  Study  of  United 
States  Air  Force  Requests  for  Pro- 
posals, Critiques,  and  Recommenda- 
tions," Washington,  DC,  December 
1969. 

^The  Air  Force  study  suggested 
assembling  experts  to  assist  the  sys- 
tem program  office  in  identifying  un- 
necessary requirements  in  the  RFP. 
The  AIA  study  suggested  a formal, 
top-level  RFP  review,  preferably  at 
the  Assistant  Secretary  level,  to  take 
place  prior  to  RFP  issuance  to  en- 
sure elimination  of  requirements 
which  trigger  excessive  effort  and 
documentation. 

*Brigadier  General  A.  L.  Esposito, 
USAF,  “An  Analysis  of  Frustrations : 
The  RFP-Proposal  Cycle,"  Defense 
Management  Journal,  January  1973, 
p.  8;  Lieutenant  General  James  T. 
Stewart,  USAF,  “Source  Selection 
Process  Faces  Winds  of  Change,”  De- 
fense Management  Journal,  January 
1973,  p.  13;  and  Steele  Morris,  “ ‘Com- 
munications Effectiveness'  Needed  in 
RFP-Proposal-Contract  Award  Cy- 
cle," Defense  Management  Journal, 
January  1973,  p.  17. 

^Headquarters  Air  Force  Systems 
Command,  Aeronautical  Systems  Di- 
vision, ASDP  800-6,  “Request  For 
Proposal  Preparation  Guide,"  An- 
drews Air  Force  Base,  MD,  May  29, 
1973. 
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Figure  1.  AFSC's  Proposed  Organization 
of  the  RFP  in  1975 


Volume  I:  Proposal  Preparation  Instructions 

Volume  II:  Model  Contract 

--  Supplies  and  Services 
--  Contract  Terms/Conditions 
--  Contract  Administration  Data 
--  DD  Form  1423 

Volume  III:  Attached  Documents  and  References 

--  Statement  of  Work 
--  Specifications 
--  Engineering  Drawings 
--  DD  Form  254 

--  Contract  Work  Breakdown  Structure 


Volume  IV: 


Classified  Parts  of  RFP,  If  Any 


would  facilitate  more  effective 
communication  with  the  poten- 
tial seller.  The  recommended 
format,  which  permitted  the 
reader  to  proceed  through  the 
request  for  proposal  in  a logical 
manner,  separated  the  RFP  into 
four  parts : 

• Volume  1:  executive  sum- 
mary (proposal  preparation  in- 
structions). 

• Volume  2:  model  contract. 

• Volume  3 : reference  docu- 
ments. 

• Volume  4:  classified  parts 
of  the  request  for  proposals. 

The  development  of  the  above 
format  appeared  to  be  the  first 
major  effort  by  any  service  or- 
ganization to  put  into  action  any 
of  the  recommendations  pertain- 
ing to  the  request  for  proposal 
problem  as  suggested  by  the  Air 
Force  and  Aerospace  Industries 
studies. 

Two  years  later,  in  May  1975, 
the  Air  Force  Systems  Command 
published  Pamphlet  70-4,  “ Re- 
quest for  Proposal  Preparation 
Guide,”  which  superseded  ASDP 


800-6  but  which  retained  much 
of  the  material  contained  in  the 
earlier  pamphlet.  The  informa- 
tion in  Pamphlet  70-4  was  ori- 
ented primarily  toward  docu- 
ments controlled  under  Air 
Force  Regulations  70-15  and 
800-2 ; however,  the  concepts  and 
procedures  could  be  tailored  for 
limited  funding  and  less  complex 
procurements. 

Pamphlet  70-4  recommended 
that  the  request  for  proposal  be 
transmitted  to  the  offeror  with  a 
brief  executive  summary  letter; 
this  letter  would  provide  top 
management,  both  in  industry 
and  government,  with  a concise 
review  of  the  salient  features  of 
the  procurement.  Additionally, 
the  pamphlet  presented  a de- 
tailed arrangement  for  organiz- 
ing the  request  for  proposal  into 
four  separate  volumes  (Figure 
1). 

The  1975  survey  research 
study  was  undertaken  to  collect 
data  on  how  industry  contract 
administrators  perceived  the  re- 
quest for  proposal.®  A question- 


naire instrument  was  developed 
for  securing  the  data.  Because 
of  cost  considerations,  no  per- 
sonal interviews  were  conducted. 

The  statements  used  in  collect- 
ing the  data  were  based  on  in- 
formation and  ideas  that  were 
gained  through  reading  requests 
for  proposals,  the  Air  Force  and 
Aerospace  Industries  Associa- 
tion studies,  and  the  Defense 
Management  Journal  articles 
dealing  with  RFP  problems. 
They  were  designed  to  determine 
whether  industry  users  perceived 
the  requests  for  proposals  in  the 
same  manner  as  those  who  had 
studied  RFPs,  had  written 
about  them,  or  were  currently 
writing  RFPs  for  procurements. 

The  instrument  was  mailed  to 
108  of  the  largest  defense  con- 
tractors. The  names  of  the  con- 
tractors were  taken  from  a list 
published  in  the  November  25, 

1974,  issue  of  Aviation  Week  and 
Space  Technology,  while  the 
names  of  contract  administra- 
tors and  marketing  vice  presi- 
dents were  found  in  the  Stan- 
dard and  Poor’s  Directory  of 
Businesses.  The  response  rate 
was  approximately  67  percent, 
which  is  considered  good  for  a 
survey  study  of  this  type.^ 

^The  study  was  suggested  by  the 
author  as  a part  of  a continuing  ef- 
fort by  the  Business  Research  Man- 
agement Center,  Wright-Patterson 
Air  Force  Base,  Ohio,  to  encourage  Air 
Force  specialists  and  academicians  to 
undertake  research  projects  dealing 
with  the  Air  Force  procurement  proc- 
ess. For  additional  information  on  this 
program  see:  Cecil  V.  Hynes,  "Air 
Force  Business  Research  Management 
Center  Offers  Research  Opportunities 
Even  For  Those  at  Civilian  Universi- 
ties," Marketing  News,  October  2U, 

1975,  p.  3. 

'’Paul  E.  Green  and  Donald  S.  Tull, 
Research  for  Marketing  Decisions 
(Englewood  Cliffs,  NJ : Prentice-Hall, 
1970),  p.  158.  For  mail  questionnaires, 
the  model  response  rate  is  on  the  or- 
der of  20  to  IfO  percent. 
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Because  of  the  manner  in 
which  the  sample  was  selected, 
the  findings  were  limited  and 
showed  only  the  perceptions  and 
opinions  of  the  individual  re- 
spondents. It  would  be  difficult 
and  probably  erroneous  to  gener- 
alize from  those  findings.  How- 
ever, the  respondents  held  re- 
sponsible positions  with  large 
defense  contractors  and  collec- 
tively represented  a wealth  of 
experience  in  dealing  with  re- 
quests for  proposals  from  indus- 
try’s view  of  the  procurement 
process. 

Another  limitation  inherent  in 
this  type  of  research  was  the 
necessity  of  making  generalized, 
and  sometimes  simplified,  state- 
ments on  the  instrument  used  to 
collect  data  without  being  able 
to  tie  the  statements  directly  to 
a specific  procurement  action. 
The  questions  presented  to  the 
respondents  were  written  in  gen- 
eral terms  that  might  appear  in 
any  request  for  proposal  for 
either  large  complex  weapon  sys- 
tems or  smaller  procurements. 
However,  because  only  the  larg- 
est contractors,  who  tended  to  be 
primary  contractors,  were  se- 
lected, it  was  anticipated  that 
the  respondents  would  represent 
firms  dealing  with  large,  com- 
plex weapons  procurements.  A 
review  of  the  findings  and  com- 
ments suggests  that  these  were 
in  fact  the  individuals  reached 
by  the  study. 

Findings 

Figure  2 on  p.  30,  which  de- 
picts the  industry  questionnaire, 
reflects  the  percentage  spread  of 
responses  to  the  questions.  The 
summation  of  responses  to  each 
question  is  portrayed  in  Figure 
3 on  p.  31. 

A review  of  the  questionnaire 


indicates  that  a majority  of  the 
respondents  agreed  with  most  of 
the  questions.  However,  many 
tended  to  be  undecided  about  or 
to  disagree  with  questions  2,  11, 
15,  18,  22,  23,  and  24. 

The  respondents  were  also 
asked  to  write  their  suggestions 
for  improving  the  requests  for 
proposals.  Several  submitted 
specific  recommendations,  of 
which  the  following  were  the 
most  significant : 

• Eliminate  the  practice  of 
adding  to  the  RFP  requirements 
for  data  that  are  not  essential  to 
the  main  purpose  of  the  RFP, 
but  in  general  appear  to  be  put 
into  the  RFP  because  some  de- 
partment requires  information 
or  needs  only  to  be  aware  of  an 
activity. 

• Eliminate  or  minimize  eval- 
uation criteria  that  promote 
gamesmanship  on  the  part  of  the 
offerors  or  gimmicks  being  offer- 
ed rather  than  hard,  cold,  con- 
servative, prudent,  and  sound 
information;  for  example,  “this 


is  what  we  can  do,”  or  “this  is 
how  we  can  do  it.” 

• Provide  adequate  advance 
notice  of  impending  RFP  re- 
lease. 

• File  early,  with  a central 
agency,  the  RFP  pages  contain- 
ing standard  contractors’  repre- 
sentations, certifications,  and  ac- 
knowledgments. The  RFP  then 
would  require  only  one  block  to 
insert  a file  number  or  similar 
identification  of  compliance. 

The  Respondents 

The  respondents  were  asked 
for  information  about  their  com- 
panies’ total  sales,  the  percent- 
age of  sales  to  the  government, 
the  title  of  their  positions,  the 
length  of  time  they  had  spent  in 
those  positions,  and  the  number 
of  years  they  had  worked  with 
requests  for  proposals.  Eleven 
chose  not  to  supply  this  informa- 
tion. 

Those  who  complied  with  the 
request  reported  sales  varying 
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Figure  2.  Industry  Request  for  Proposal  Questionnaire 

Code:  Strongly  Agree  (SA);  Agree  (A);  Undecided  (UNO), 

Disagree  (DA);  Strongly  Disagree  (SD) 

Question 

Responses 

(number) 

SA 

UNO 

(percent) 

DA 

SD 

The  procurement  office  should  provide  an  "Executive  Summary"  of  the 
Request  for  Proposal  which  would  be  a concise  summary  of  the  salient 
features  of  the  procurement. 

71 

45 

41 

3 

8 

3 

The  number  of  pages  in  a Request  for  Proposal  should  be  limited. 

73 

12 

42 

8 

36 

2 

The  number  of  pages  in  the  contractor's  response  for  technical  proposals 
should  be  limited. 

72 

19 

49 

1 

28 

3 

The  number  of  pages  in  the  contractor's  response  for  management 
proposals  should  be  limited. 

72 

26 

51 

0 

23 

0 

The  procurement  officer  responsible  for  source  selection  on  complex 
weapon  system  procurements  should  visit  each  offeror's  facilities  before 
source  selection  is  made. 

70 

31 

41 

10 

13 

5 

The  data  requested  by  the  Request  for  Proposals  from  offerors  should 
only  be  requested  if  it  helps  the  source  selection  authority  to  make  a 
better  decision. 

72 

47 

50 

0 

3 

0 

The  material  in  a Request  for  Proposal  should  be  organized  and  arranged 
so  the  bidder  gets  the  story  in  a logical  step-by-step  fashion. 

72 

51 

49 

0 

0 

0 

The  Request  for  Proposal  should  be  very  explicit  in  explaining  to  the 
bidder  all  concepts,  requirements,  and  risks  in  the  proposed  procurement. 

69 

41 

38 

8 

13 

0 

The  objectives  of  the  technical  and  management  approaches  should  be 
clearly  defined  in  the  Request  for  Proposal. 

73 

48 

45 

4 

3 

0 

Unrealistic  requirements  should  not  be  included  in  a Request  for  Proposal. 

72 

65 

28 

4 

3 

0 

A standard  outline  should  be  developed  for  writing  all  Requests  for 
Proposals. 

73 

16 

37 

15 

26 

6 

When  bidders  are  required  to  make  trade-off  studies  to  select  the  best 
approaches,  the  relative  importance  of  such  factors  as  weight,  performance, 
reliability,  development  time,  risk,  maintenance,  and  costs  should  be 
defined  in  the  Request  for  Proposal. 

73 

48 

44 

1 

6 

1 

The  Request  for  Proposal  should  give  firm  criteria  for  proposal  evaluation, 
as  well  as  the  actual  or  appropriate  weighting  factor  for  each  of  the  criteria 
or  group  of  criteria. 

73 

48 

47 

3 

1 

1 

The  Request  for  Proposal  should  specify  the  degree  to  which  each 
referenced  document  (ASPR,  DoD  regulations.  Air  Force  regulations,  etc.) 
is  applicable  in  the  contract  and  the  degree  to  which  it  is  to  be  discussed 
in  the  proposal. 

70 

27 

47 

12 

14 

0 

The  Request  for  Proposal  should  provide  a definitive  proposal  outline 
that  should  be  followed  by  all  bidders. 

72 

21 

28 

13 

37 

1 

Before  sending  the  Request  for  Proposal  to  bidders  it  should  be  read  by 
objective  nonparticipating  reviewers  who  would  take  the  bidder's  viewpoint 
and  recommend  changes  that  would  improve  communication. 

73 

27 

58 

7 

4 

4 

The  source  selection  authority  should  rely  primarily  upon  existing  evalua- 
tions of  contractor's  procedural  systems,  and  upon  direct  inspections  at 
contractors'  facilities  in  lieu  of  requesting  voluminous  plans. 

73 

29 

53 

9 

7 

2 

The  Request  for  Proposal  should  allow  offerors  more  time  for  response. 

69 

17 

28 

28 

28 

0 

The  time  allowed  for  response  to  the  Request  for  Proposal  should  be 
related  to  the  complexity  of  the  procurement. 

73 

37 

56 

1 

4 

1 

A minimum  response  period  for  any  weapon  system  Request  for  Proposal 
should  be: 

68  ( 30  days)(60  days 

12  26 

H90  days)(120  daysHmore) 
38  16  7 

The  sections  of  the  ASPR  dealing  with  Request  for  Proposal  should  be 
reviewed  by  a committee  of  government  and  industry  procurement 
specialists  and  rewritten  to  reflect  the  present  procurement  environment 
and  lessons  learned  in  the  past  few  years. 

72 

38 

40 

15 

7 

0 

Generally  the  ASPR  is  too  constrictive  as  it  tries  to  insure  impartiality  with 
the  result  of  sacrificing  efficiency  in  the  procurement  process. 

49 

31 

37 

32 

0 

0 

Requests  for  Proposals  for  non-complex  weapon  systems  are  easily  under- 
stood and  therefore  no  changes  are  recommended. 

69 

2 

15 

26 

44 

14 

A well-written,  brief  Request  for  Proposal  is  the  primary  key  to  improving 
the  government's  procurement  process. 

71 

14 

39 

20 

23 

4 

30 
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Figure  3.  Summation  of  Responses  to  Each  Question  in  the 
Request  for  Proposal  Study 


Disagree 


Undecided 


Agree 


Strongly 
Agree  I 
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selection  was  small  and  based  on  instrument  by  which  the  mes- 


from  a low  of  $200  million  to  a 
high  of  $31.5  billion.  The  median 
sales  figures  for  the  firms  were 
$250  million ; the  median  sales  to 
the  government  were  $108.9  mil- 
lion, or  43.5  percent  of  the  med- 
ian total  sales. 

The  average  length  of  time  the 
respondents  had  spent  in  their 
positions  was  6 years.  Only  three 
had  held  the  same  posts  for  over 
15  years. 

The  respondents  as  a group 
had  spent  a total  of  1,209  years 
working  with  requests  for  pro- 
posals, an  average  of  19  years  per 
person.  Clearly  they  had  had  am- 
ple experience  working  with 
RFPs,  and  that  experience  had 
qualified  them  for  responsible 
positions.  Twenty-three  respond- 
ents were  vice  presidents  (execu- 
tive vice  presidents,  or  vice  pres- 
idents of  marketing,  finance,  or 
contract  administration),  twenty 
were  directors  of  contracts  or 
contract  administration,  and 
eighteen  were  marketing  and 
sales  managers.  All  of  this  sug- 
gests that  their  opinions  carried 
some  weight. 

Summary  and  Conclusions 

The  foregoing  review  of  in- 
dustry contract  administrators’ 
perceptions  relating  to  the  de- 
velopment and  use  of  the  request 
for  proposal  points  up  the  fact 
that  the  majority  of  the  respond- 
ents agreed  with  the  statements 
presented  to  them,  which  sug- 
gests that  they  agreed  with 
many  of  the  recommendations 
made  in  the  1969  Air  Force  and 
Aerospace  Industries  Associa- 
tion studies  and  the  suggestions 
for  improving  the  request  for 
proposal  that  were  presented  in 
the  January  1973  issue  of  the 
Defense  Management  Journal. 

The  findings  were  admittedly 
limited,  inasmuch  as  the  sample 


judgment,  and  presumably  those 
with  strong  feelings  regarding 
the  requests  for  proposals  were 
the  ones  who  responded  and  of- 
fered additional  comments.  This 
may  have  biased  some  of  the  find- 
ings, yet  it  should  be  reiterated 
that  the  respondents,  collective- 
ly, represented  impressive  pro- 
fessional experience  in  dealing 
with  the  request  for  proposals 
from  industry’s  side  of  the  pro- 
curement process. 

Despite  the  study’s  obvious 
limitations,  the  findings  should 
provide  procurement  policy  mak- 
ers with  new  insights  and  infor- 
mation regarding  RFPs.  Simply 
from  a communications  stand- 
point, it  is  helpful  to  understand 
how  the  recipient  of  the  message 
perceives  the  message  and  the 

should  be  noted  that  recent  De- 
partment of  Defense  initiatives  ad- 
dress certain  points  made  previously 
by  industrialists.  Department  of  De- 
fense Directive  4105.62,  “Selection  of 
Contractual  Sources  for  Major  De- 
fense Systems,”  January  6,  1976,  ad- 
dresses RFP  feedback,  minimum  ex- 
pense, and  model  contracts.  DoDD 
5000.2,  “Major  System  Acquisition 
Process,”  January  8,  1977,  says  that 
information  requested  should  be  lim- 
ited to  that  essential  to  the  contract, 
and  that  the  contractor’s  existing 
data  should  be  utilized  whenever  pos- 
sible. 


sage  is  conveyed.  These  percep- 
tions by  industry  of  the  request 
for  proposal  might  provide  the 
motivation  for  a new  look  at  this 
document  with  a view  to  making 
it  a better  means  of  communica- 
tion.® 

Finally,  the  study  might  serve 
as  the  basis  for  developing  fu- 
ture research  pertaining  to  the 
request  for  proposal  and  other 
elements  which  are  essential  to 
the  government  procurement 
process.  DMJ 
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On  the  Effects  of  Modularity 
on  Operating  and  Support  Costs 


Our  'pseudonymous  author,  rumored  to  be  an  analyst  somewhere 
in  DoD,  finds  that  a little  inspiration  can  lead  to  some  rather 
farfetched  notions. 


I t was  early  evening  in  the  dimly  lit  cocktail 
lounge,  and  I was  relaxing  after  a long  day  by 
nursing  a parched  martini,  the  pair  of  olives 
nestled  in  its  depths  blankly  staring  back  at  me. 
The  hectic  nature  of  the  day  had  nearly  im- 
mobilized my  body,  but  had  inspired  my  brain 
to  frantic  activity. 

I had  just  attended  the  first  day’s  sessions  of 
an  operating  and  support  costs  symposium.  The 
presentations  had  been  highly  thought-provok- 
ing. Letting  my  mind  wander  back  to  the  previ- 
ous day,  when  Dr.  Marco  Fiorello  and  I had 
discussed  various  issues  involved  in  modularity, 
I coalesced  that  conversation  with  the  more 
stimulating  points  of  the  symposium  and  con- 
ceived three  sharply  different  approaches  to 
aircraft  avionics  modularization. 

GALRU 

In  my  first  case,  all  of  the  avionics  are  in  one 
Giant  Avionics  Line  Replaceable  Unit  (as  is 
the  bureaucratic  wont,  we’ll  call  it  a GALRU). 
Naturally,  a few  physical  difficulties  might  be 
encountered ; for  instance,  a crane  would  prob- 
ably be  necessary  standard  ground  equipment 
to  install  and  remove  the  GALRU  (in  the  inter- 
ests of  cost-effectiveness,  the  crane  could  do 
double  duty  on  other  jobs  as  well).  There  might 
also  be  a center  of  gravity  problem,  but  we’ll 

by  Rustle  Em  Jenay 
DoD  Analyst? 


presume  that  it  could  be  overcome  by  our  air- 
craft design  people,  who  typically  seem  to  be 
in  search  of  a challenge. 

With  but  a single  LRU  to  malfunction,  flight- 
line diagnosticians  wouldn’t  be  bothered  with 
pulling  the  wrong  unit.  This  would  presumably 
reduce  false  removals  to  a very  low  rate.  And 
integrated  avionics  would  now  come  into  its 
own. 

Certain  life  cycle  cost  enthusiasts,  hung  up 
on  introducing  new  LRUs  into  the  inventory, 
would  predictably  anathematize  the  GALRU. 
But  limiting  the  LRUs  to  one  per  new  aircraft 
would  substantially  reduce  such  problems  as 
LRU  management,  assignment  of  Federal  Stock 
Numbers,  and  so  on.  The  number  of  technical 
orders  would  also  be  trimmed  to  one,  although 
it  would  be  27  feet  thick  (wheels  and  a motor 
could  be  attached  to  its  cover  to  make  it  suffi- 
ciently mobile).  Costly  hot  mock-ups  in  the  field 
would  be  avoided  since  the  GALRU  would  serve 
as  its  own  hot  mock-up.  Finally,  those  taxing 
decisions  on  which  LRUs  to  modify  would  be 
eliminated,  there  being  only  one  LRU,  and  a 
significant  amount  of  study  money  and  modifi- 
cation board  time  would  be  saved. 

Of  course,  there  might  be  a minor  drawback 
or  two.  Since  any  failure  would  require  removal 
of  all  the  avionics  from  the  aircraft,  there  would 
inevitably  be  quite  a few  of  these  giant  LRUs 
being  shuffled  between  the  flight  line  and  the 
intermediate-level  shop,  which  might  prove  to 
be  a trifle  expensive  in  terms  of  spares.  How- 
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ever,  proper  application  of  design  to  cost  could 
keep  this  under  control. 

While  100  percent  field-level  repair  of  the  LRU 
is  theoretically  possible  using  Shop  Replaceable 
Units  from  the  depot,  in  actuality,  with  complex 
avionics  some  of  the  LRUs  are  always  NRTS’d 
back  to  the  depot  for  repair  for  one  reason  or 
another.  Again,  stocking  this  pipeline  could  be 
expensive. 

Those  people  promoting  avionics  standardiza- 
tion across  various  types  of  aircraft  would  no 
doubt  tend  to  frown  on  the  GALRU,  but  Re- 
liability Improvement  Warranty  proponents 
would  be  delighted  in  that  one  RIW  would  cover 
all.  Some  analysis  would  be  needed  to  verify  the 
GALRU  concept,  but  the  fact  that  there  is  only 
one  LRU  should  simplify  this  process  enough  to 
compensate  for  the  less-consequential  problem 
of  having  5,000  SRUs. 

FMB 

As  you  have  probably  anticipated,  in  our 
second  case  every  resistor,  transistor,  inte- 
grated circuit,  etc.,  is  an  LRU.  The  biggest 
problem  here  is  that  there  would  have  to  be  as 
many  connectors  as  there  are  discrj^e  compo- 
nents, not  to  mention  the  mass  of  in*rconnect- 


ing  wires.  One  might  initially  conclude  that 
5,000  pounds  of  connectors  is  an  insurmount- 
able problem,  especially  since  there  must  be 
easy  flightline  access  to  each  LRU,  i.e.,  compo- 
nent. Obviously,  surface  area  would  have  to  be 
rather  extensive. 

No  problem  for  the  imaginative  government 
manager : we  simply  turn  these  difficulties  into 
an  asset  by  replacing  the  aircraft  skin  and 
structure  with  one  huge  mother  board.  The 
monocoque  construction  would  be  the  absolute 
last  word  in  avionics-structure  integration. 
Plugged  into  the  stressed-skin  mother  board 
would  be  the  127,532  LRUs.  Should  there  prove 
to  be  insufficient  surface  on  the  inside  of  the 
aircraft,  the  LRUs  could  also  cover  its  outside. 
Significantly,  this  would  save  money  by  elimi- 
nating any  requirement  for  speed  brakes  or 
vortex  generators. 

The  standardization  folks  would  be  extremely 
pleased  with  the  FMB,  since  standardization 
would  be  completely  captured  at  the  LRU  level 
without  infringing  on  design  freedom. 

I will  admit  that  it  may  take  a little  longer  to 
find  the  right  LRU  to  replace  on  the  flight  line 
since  there  are  127,532  from  which  to  choose, 
but  this  would  certainly  be  more  than  offset  by 
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the  complete  elimination  of  intermediate-  and 
depot-level  maintenance.  Just  think  how  much 
could  be  saved  by  closing  depots  and  excluding 
all  intermediate  and  depot  automated  ground 
equipment ! And  SRUs  would  be  eliminated  for- 
ever! 

Another  enormous  benefit  would  be  the  elimi- 
nation of  all  the  expense  and  bother  of  optimum 
repair-level  analysis,  as  there  would  be  only  one 
level  of  repair.  We  would  also  have  achieved 
the  impossible  dream  of  all  true  R&Ders:  100 
percent  throw-away  LRUs  (thank  goodness  you 
can’t  fix  a resistor).  While  there  would  probably 
be  corrosion  problems  when  dealing  with  127,- 
532  connectors,  space  limitations  in  this  article 
preclude  further  discussion  of  this  topic  at  this 
time. 

Since  there  isn’t  a computer  simulation  pro- 
gram that  can  handle  127,532  LRUs,  analysis  to 
verify  the  flying  mother  board  concept  would 
have  to  be  done  by  hand. 

GALRUFMBC 

For  those  pessimists  not  sold  on  either  of  the 
above  alternatives,  there  is  a middle  ground — 
the  GALRUFMBC  (the  C standing  for  com- 
promise). The  trick  is  to  take  the  best  features 
of  each  extreme  while  avoiding  the  bad.  This 
would  necessarily  involve  some  compromises 
and  trade-offs  requiring  study,  analysis,  and 
even  hard  thinking — not  to  mention  voluminous 
reports,  lengthy  meetings,  and  state-of-the-art 
seminars.  But  who  is  better  suited  to  such  a task 
than  a professional  bureaucrat?  The  crane  re- 
quired for  the  GALRU  might  be  a,  difficult 
notion  to  sell,  so  some  upper  limit  on  LRU 
weight  would  be  essential.  With  women  now 
participating  more  fully  in  maintenance  tasks 
and  some  of  us  old  men  having  weak  backs,  this 
limit  would  be  set  at  30  pounds.  The  potential 
center  of  gravity  problem  on  the  GALRU  thus 
disappears  as  the  LRUs  can  be  distributed 
around  the  aircraft  a bit  (but  500  pounds  should 
be  added  to  account  for  interconnecting  cables). 

Because  there  would  now  be  more  than  one 
LRU  to  pull,  we  would  need  to  be  concerned 
with  pulling  the  right  one  most  of  the  time.  As 
the  number  of  LRUs  increased,  high  diagnostic 
error  rates  would  become  less  tolerable.  Inte- 
gration and  software  problems  between  LRUs 
would  need  careful  attention.  Hot  mock-ups 
would  probably  have  to  be  provided  to  the  field 


so  that  those  personnel  wouldn’t  have  to  sneak 
one  together  under  the  cover  of  darkness. 

By  adroitly  arranging  LRUs  by  functional 
characteristics  and  failure  rates,  diagnostic 
errors  could  be  reduced,  and  pipelines  would  not 
be  filled  with  loads  of  nonfailed  and  expensive 
material.  The  flying  mother  board  advantage  of 
standardization  could  be  partially  retained  if 
the  functional  arrangement  of  LRUs  was  so 
ingenious  that  it  even  considered  multiple  air- 
craft-type applications.  Lost  in  the  compromise, 
however,  would  be  throw-away  LRUs.  Finally, 
we  would  have  to  retain  optimum  repair-level 
analysis,  not  to  mention  depots. 

But  then  being  an  analyst  by  profession  my- 
self and  having  a distinct  inclination  to  stay 
one,  I generally  prefer  the  compromise  position, 
which  coincidentally  would  require  the  use  of 
my  skills. 
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Relating  Promised  Performance 
To  Military  Worth: 

An  Evaluating  Mechanism 


A multi-attribute  utility  evaluation  procedure  is  successfully 
applied  to  the  Navy’s  Electronic  Warfare  Suite. 


Edito'ds  Note:  This  article 
is  an  abridged  version  of 
a technical  report  describ- 
ing the  multi-attribute  util- 
ity approach  and  its  appli- 
cation to  a major  military 
system.  See  Michael  L. 
Hays,  Michael  F.  O’Connor, 
and  Cameron  R.  Preston, 
“An  Application  of  Multi- 
Attribute  Utility  Theory: 
Design-to-Cost  Evaluation 
of  the  U.  S.  Navy’s  Elec- 
tronic Warfare  System,” 
Technical  Report  DT/TR 
75-3,  Decisions  and  De- 
signs, Inc.,  McLean,  VA, 
October  1975. 


c 

urrent  Design  to  Cost  acqui- 
sition policy  implies  the  need  for 


explicitly  relating  performance 
to  military  worth  within  a cost 
constraint  as  a means  of  evalu- 
ating contractors’  design  pro- 
posals for  major  military  sys- 
tems. A multi-attribute  utility 
evaluating  procedure  addressing 
this  requirement  was  one  result 
of  research  sponsored  by  the  De- 
fense Advanced  Research  Proj- 
ects Agency,  monitored  by  the 
Office  of  Naval  Research,  De- 
partment of  the  Navy,  and  di- 
rected toward  developing,  apply- 
ing, and  transferring  to  users  in 
the  Department  of  Defense  ad- 
vanced management  technologies 
for  decision  making.  This  article 
describes  the  application  of  this 
MAU  evaluating  procedure  to 
the  evaluation  of  contractors’ 
proposals  for  the  Navy’s  Elec- 
tronic Warfare  System,  a group 
of  sophisticated  and  complex. 


semiautomated,  computerized 
suites  for  detecting  and  respond- 
ing to  different  threats  to  differ- 
ent naval  vessels  in  different 
situations.  This  article  details 
the  MAU  model  used  to  evaluate 
the  Navy’s  Electronic  Warfare 
System  design  proposals  as  an 
illustration  of  a new  and  success- 
ful approach  to  evaluating  major 
military  systems  under  current 
DTC  acquisition  policy. 

Background 

Faced  with  tightening  budgets 
and  increasing  costs  in  the  acqui- 
sition of  major  military  systems, 
DoD  promulgated  in  1971  a DTC 
acquisition  policy  linking  de- 
signed performance  and  other 
benefits  to  cost  ceilings.  This 
policy  is  not  another  formal  pro- 
nouncement reemphasizing  the 
importance  of  performance  on 
the  one  hand  and  economy  on  the 
other.  Instead,  Defense  Depart- 
ment directives  enunciating  DTC 
policy  integrate  both  concerns, 
with  significant  implications  for 
the  procurement  cycle  of  design, 
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production,  and  scheduling.^  In 
doing  so,  DTC  policy  attaches 
great  importance  to  design  by 
making  cost  a design  parameter 
and  stressing  the  need  for  trade- 
offs between  design  and  system 
capability. 

Reflecting  the  central  tenets  of 
DTC  policy  are  the  cost-per-unit 
ceiling  and  the  general  absence 
of  performance  requirements  in 
requests  for  proposals.  By  stip- 
ulating a cost-per-unit  ceiling, 
DTC  RFPs  make  cost  overruns 
and  insufficient  acquisitions  less 
likely.  Because  of  this  ceiling, 
costly  improvements  in  one  area 
of  system  capability  must  be  bal- 
anced against  possible,  econom- 
ically mandated  deterioration  in 
another.  Since  performance  re- 
quirements may  require  such  im- 
provements, properly  stated 
DTC  RFPs  minimize  or  elimi- 
nate performance  requirements 
and  give  contractors  considera- 
ble latitude  in  designing  systems 
for  certain  missions  in  certain 
scenarios.  As  a result,  they  chal- 
lenge contractors  to  design  sys- 
tems maximizing  the  military 
worth,  or  benefit,  of  their  total 
performance  within  a cost  con- 
straint. Since  system  designs  re- 
flect contractors’  different  cor- 
porate, industrial,  and  techno- 
logical capabilities  and  interests, 
and  their  different  interpreta- 
tions of  the  missions  of  the  sys- 
tem, contractors  are  likely  to 
propose  highly  diverse  systems, 
each  promising  to  satisfy  the 
military  service’s  needs. 

Evaluating  Design  Proposals 

The  cost-per-unit  ceiling  and 
the  general  absence  of  perform- 
ance requirements  make  evalu- 
ating diverse  design  proposals 
more  difficult  than  it  has  been  in 
the  past,  since  all  contractors  are 
likely  to  propose  designs  meeting 


the  cost-per-unit  ceiling.  Para- 
doxically, for  those  designs  pro- 
posing and  then  estimated  to 
meet  the  cost-per-unit  ceiling, 
cost  itself  is  no  longer  a signifi- 
cant consideration  in  evaluating 
the  proposals.  Moreover,  RFPs 
stating  missions  and  scenarios 
rather  than  performance  re- 
quirements alter  the  focus  of  the 
evaluation  process  from  a direct 
consideration  of  system  capabili- 
ties to  an  analysis  of  the  military 
worth  of  the  total  performance 
of  the  system. 

Accordingly,  the  military 
service  confronts  the  crucially 
important  and  extremely  com- 
plex problem  of  evaluating  di- 
verse design  proposals  without 
employing  the  customary  cri- 
teria of  cost  or  common  per- 
formance requirements.  The  log- 
ic of  evaluation  and  the  nature 
of  competitive  bidding  require 
that  any  program  for  evaluating 
the  military  worth  of  proposed 
systems  must  be  carefully  con- 
structed to  meet  four  criteria; 
namely,  it  must  discriminate  ef- 
fectively among  the  alternative 
design  proposals,  and  it  must  be 
reliable,  intelligible,  and  equita- 
ble. Any  program  meeting  these 
criteria  has  but  one  operational 
purpose:  to  relate  measures  of 
performance  of  the  controllable 
parameters  of  a design  proposal 
to  the  total  military  worth  of  the 
proposed  system.  Only  after  such 
an  evaluation  does  the  service 
need  to  estimate  the  contractors’ 
abilities  to  meet  the  cost-per- 
unit  ceiling. 

The  first  two  criteria,  discrim- 
ination and  reliability,  are  satis- 
fied or  not  by  the  nature  of  the 

^Department  of  Defense  Directive 
5000.1,  “Major  System  Acquisitions,” 
July  13,  1971  (latest  version  issued 
January  18,  1977);  and  DoD  Direc- 
tive 5000.28,  “Design  to  Cost,"  May 
23, 1975. 


Aspects  of  the  Model 
Structuring 

The  many  considerations  addressed 
in  developing  an  MAD  model  are  fi- 
nally organized  hierarchically;  more 
general  elements  are  successively 
decomposed  until  a structure  is 
developed  that  aggregates  measures 
of  performance  characteristics  up 
through  TSU  of  each  design  proposal 
or  that  justifies  any  higher-level 
figure  of  utility  or  effectiveness  by 
reference  to  lower-level  figures.  For 
these  reasons,  correct  structuring  of 
the  problem  is  the  most  important 
step  in  the  process.  Moreover,  cor- 
rect structuring  helps  to  reduce 
possible  errors  in  judgment  and  to 
ensure  that  variations  in  the  design 
proposals  are  reflected  in  different 
figures  of  TSU. 

Utility  and  Effectiveness 

The  words  "utility"  and  "effective- 
ness" are  used  in  this  article  in 
senses  consistent  with,  though  not 
necessarily  so  specific  as,  their  usage 
in  related  disciplines.  "Utility"  is 
mainly  applied  to  evaluation  (in  the 
narrow  sense),  "effectiveness"  to 
prediction,  and  each  has  its  impor- 
tant part  in  evaluation  (in  the  broad 
sense).  The  distinction,  admittedly 
slight  (see  Figure  1 for  the  area  in 
which  simulation  runs  may  or  may 
not  be  used),  is  that  between  deter- 
minations involving  military  trade- 
offs and  those  involving  technical 
system  capabilities. 

Scaling 

Since  an  MAU  model  involves  many 
different  kinds  of  elements  and 
many  different  units  of  measure- 
ment, it  is  important  to  have  a 
common  unit  of  measurement  that 
makes  all  measures  commensurable 
to  facilitate  aggregation  up  through 
the  model.  The  convention  is  to 
scale  either  utility  or  effectiveness 
from  zero  to  100.  Accordingly, 
various  performance  characteristics 
may  be  measured  as  percentages 
which  become  amenable  to  aggrega- 
tion at  higher  levels  in  the  MAU 
model. 
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model  for  evaluating  design  pro- 
posals. The  third  criterion,  in- 
telligibility, is  ensured  by  a 
structure  that  makes  explicit  the 
paths  from  the  data  to  the  con- 
clusion of  the  evaluation.  The 
fourth  criterion,  equity,  is  pro- 
moted by  an  evaluating  mecha- 
nism that  is  comprehensive  in  in- 
cluding all  possible  considera- 
tions known  to  be  relevant  prior 
to  evaluating  a design  proposal 
and  is  flexible  in  allowing  the 
incorporation  of  any  unforeseen 
but  relevant  factors  into  the 
evaluating  mechanism.  Both  cri- 
teria ensure  that  arbitrary  stip- 
ulations do  not  bias  the  evaluat- 
ing mechanism  against  distinc- 
tive design  proposals  allowed  by 
the  latitude  of  the  DTC  RFPs. 
An  evaluating  mechanism  meet- 
ing these  standards  permits  the 
military  service  to  justify  its 
selection  or  contractors  to  chal- 
lenge it  on  a rational  and  con- 
structive basis. 

Implementing  DTC 

To  meet  these  demanding  cri- 
teria under  DTC  acquisition  pol- 
icy, research  was  undertaken  by 
a private  consulting  firm  - to  de- 
velop an  evaluating  mechanism 
suitable  for  application  to  a cur- 
rent problem  in  evaluation.  Be- 
cause the  resulting  MAU  model 
for  evaluating  the  Navy’s  EW 
System  marks  the  first  imple- 
mentation of  DTC  acquisition 
policy  in  the  design  phase  of  the 
procurement  cycle,  a brief  ac- 
count of  its  origins  and  use  is 
appropriate. 

^^There  are  essentially 
two  kinds  of  ap- 
proaches to  evaluation: 
global  and  decom- 
posed/' 

Although  the  MAU  model  was 


employed  throughout  the  entire 
evaluation,  it  was  most  fully  de- 
veloped and  most  extensively  ap- 
plied in  the  evaluation  only  of 
technical  systems  utility  (TSU) 
of  the  suites  and  of  the  EW  Sys- 
tem as  a whole.  (Other  elements 
of  military  worth  or  of  cost  and 
management  were  evaluated 
somewhat  differently.)  The  MAU 
model  for  TSU,  to  which  this 
article  confines  itself,  was  devel- 
oped during  several  working  ses- 
sions between  Navy  experts  and 
analysts  of  the  consulting  firm. 
Navy  experts  extrapolated  data 
for  performance  from  each  de- 
sign proposal,  incorporated  those 
data  if  they  seemed  reasonable, 
or  amended  them  if  they  did  not. 
The  model  established  that  two 
quite  different  design  proposals 
taken  as  wholes  were  of  approxi- 
mately equal  military  worth  and 
were  clearly  superior  to  four 
others.  As  a result,  the  contrac- 
tors offering  them  were  selected 
for  and  have  now  undertaken 
further  engineering  development 
for  prototype  testing.  The  use  of 
the  MAU  model  has  not  been 
limited  to  this  stage  in  the  pro- 
curement cycle,  however.  Modi- 
fications in  the  original  model 
have  led  to  a revised  version  that 
will  be  used  to  evaluate  and  select 
the  better  of  the  two  prototypes 
when  they  are  tested. 

The  two  questions  addressed 
by  a methodology  for  evaluating 
TSU  of  design  proposals  which 
meets  the  four  criteria  are : 
What  is  the  actual  performance 
of  a system  built  according  to  its 
design  proposal  likely  to  be?  And 
what  is  that  actual  performance 
worth?  The  first  question  re- 
quires a prediction  as  an  answer, 
the  second  an  evaluation.  To  pre- 
dict performance  of  proposed 
systems,  it  would  be  ideal  to  have 
a clairvoyant  capable  of  looking 
into  the  future  and  observing  the 


systems’  technical  effectiveness. 
Since  no  such  clairvoyant  exists, 
the  problem  becomes  one  of  mak- 
ing the  best-informed  and  best- 
reasoned  predictions  possible.  To 
evaluate  the  predicted  perform- 
ance of  such  systems,  it  would 
be  ideal  to  have  a permanent  and 
absolute  standard  of  value.  In 
lieu  of  one,  the  problem  becomes 
that  of  making  the  best  judg- 
ments possible. 


"Conditioning  varia- 
bles constitute  an  ex- 
plicit recognition  that 
the  utility  or  effective- 
ness of  the  same  meas- 
ure of  performance 
may  change  wth  a 
change  in  the  context 
of  that  performance." 

The  possible  methodologies  for 
system  evaluation  all  involve  in 
one  way  or  another  both  ques- 
tions. Those  questions  requiring 
a prediction  should  be  answered 
separately  from  questions  requir- 
ing an  evaluation.  Answers  to 
questions  requ’ring  a prediction 
concern  the  effectiveness  of  the 
proposed  system ; answers  to 
questions  requiring  an  evaluation 
concern  the  utility  of  the  pro- 
posed system.  These  distinctions 
between  prediction  and  evalua- 
tion, and  between  effectiveness 
and  utility,  will  be  observed 
throughout  the  rest  of  this  arti- 
cle. However,  the  problem  to 
which  such  methodologies  ad- 
dress themselves  will  be  regard- 
ed as  one  of  evaluation  in  a 
broader  sense. 

Prediction  and  Evaluation 

There  are  essentially  two 
kinds  of  approaches  to  predic- 
tion : 

‘Decisions  and  Designs,  Inc.,  Mc- 
Lean, VA. 
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• Personal  assessments  rang- 
ing from  guesses  to  expert  opin- 
ion. 

• Predictive  models. 

The  line  between  the  two  is  not 
so  sharp  that  models  may  not  be 
biased  in  one  way  or  another, 
and  is  further  blurred  by  vague- 
ness about  the  point  at  which  a 
proposal  becomes  too  complex 
for  a simple  personal  assess- 
ment. Since  most  evaluation 
problems  in  the  procurement  of 
military  systems  are  very  com- 
plex, however,  predictive  models, 
because  they  pool  the  knowledge 
of  many  experts,  are  preferred 
sources  of  predictions.  Of  these, 
simulation  models  are  commonly 
the  most  effectual  means  of  pre- 
dicting the  effectiveness  of  pro- 
posed systems.  The  impressive- 
ness of  simulation  runs  in  pre- 
dicting performances,  however, 
is  no  substitute  for  evaluating 
those  performances.  Since  pre- 
diction is  quite  different  from 
evaluation,  no  amount  of  per- 
formance data  can  yield  a judg- 
ment of  even  so  simple  a kind  as 
“more  is  better”  without  involv- 
ing, as  in  this  instance,  a pre- 
scriptive assertion.  The  point 
needs  to  be  emphasized  because 
simulation  runs,  while  often  de- 
sirable in  any  evaluation  of  sys- 
tem design  proposals,  are  not 
themselves  a possible  methodol- 
ogy for  system  evaluation  in  the 
broad  sense  of  the  term ; rather, 
they  are  a means  of  making  pre- 
dictions of  performance  charac- 
teristics. 

^'The  impressiveness  of 
simulation  runs  in  pre- 
dicting performances, 
however,  is  no  substi- 
tute for  evaluating 
those  performances." 

There  are  essentially  two 
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kinds  of  approaches  to  evalua- 
tion : global  and  decomposed.  The 
line  between  the  two  is  not  al- 
ways distinct  because  most 
global  judgments  result  from  a 
few  judgments  of  lesser  scope. 
The  difference  between  them, 
though  theoretically  one  of  de- 
gree, is  usually  clear  enough  to 
separate  those  approaches  that 
are  general  and  implicit  from 
those  that  are  specific  and  ex- 
plicit. 

A global  approach  to  evalua- 
tion consists  of  global  j udgments 
which  are  ratings  of  an  entire 
system  (or  an  entire  subsystem) 
on  one  or  more  scales  and  are 
often  accompanied  by  written 
explanations.  Because  the  scope 
of  global  judgments  is  very 
large,  there  is  usually  a broad 
intuitive  leap  from  information 
to  judgment.  Nevertheless,  in 
evaluating  systems  or  subsys- 
tems that  are  not  complex,  a 
global  approach  may  be  entirely 
adequate  and  appropriate. 

The  difficulties  of  the  global 
approach  arise  most  noticeably 
in  cases  involving  the  evaluation 
of  complex  proposals.  These  in- 
clude the  inability  to  use  all  the 
information  available,  the  insta- 
bility of  evaluating  standards, 
and  a variety  of  biases.  As  a gen- 
eral rule,  the  more  heterogen- 
eous the  information,  the  lower 
the  quality  of  the  global  judg- 
ment. Unfortunately,  aids  for  re- 
ducing (or  eliminating)  the  im- 
pact of  these  difficulties  have 
been  suggested  but  have  not  yet 
proven  themselves  effective.® 

A global  approach  to  evalua- 
tion of  complex  design  proposals, 
then,  is  not  presently  bound  to 
satisfy  at  least  three  of  the  four 
criteria  for  an  evaluating  mech- 
anism. Although  it  can  discrimi- 
nate among  design  proposals,  it 
cannot  ensure  reliability  or  in- 
telligibility. Accordingly,  there 


can  be  no  assurance  of  its  equity. 
These  difficulties  ultimately  re- 
sult from  the  broad  intuitive  leap 
from  information  to  judgment. 


"The  two  questions  ad- 
dressed by  a methodol- 
ogy for  evaluating  TSU 
of  design  proposals 
which  meets  the  four 
criteria  are:  What  is 
the  actual  performance 
of  a system  built  ac- 
cording to  its  design 
proposal  likely  to  be? 
And  what  is  that  actual 
performance  worth?" 

The  decomposed  approach,  on 
the  contrary,  is  a means  of  sys- 
tematically combining  informa- 
tion and  specific  judgments  to 
produce  an  evaluation  of  a de- 
sign proposal.  One  such  ap- 
proach is  known  as  a multi- 
attribute utility  approach ; its 
central  principle  is  to  address  an 
evaluation  problem  by  decompos- 
ing all  the  elements  relevant  to 
it  into  the  smallest  possible  sub- 
elements  and  then  to  combine 
them,  first  into  increasingly  ag- 
gregated measures  of  effective- 
ness based  upon  predicted  per- 
formance characteristics  and 
then  into  increasingly  aggre- 
gated estimates  of  utilities. 

The  Methodology  Selected 

To  evaluate  TSU  of  the  EW 
System  design  proposals,  the 
Navy  preferred  a model  using  a 
decomposed  approach  supported 
by  simulation  runs  to  a model 

“See,  for  example,  Paul  Slavic  and 
Sarah  Lichtenstein,  "Comparison  of 
Bayesian  and  Regression  Approaches 
to  the  Study  of  Information  Process- 
ing in  Judgment,"  Organizational  Be- 
havior and  Human  Performance,  No- 
vember 1971,  pp.  6U9-7UU. 
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Figure  1.  Two  Possible  Approaches 
to  Complex  Evaluations 


Global 


Decomposed 


using  a global  approach  support- 
ed by  simulation  runs  (see  Fig- 
ure 1). 

One  of  the  essential  differ- 
ences between  them  is  that  the 
global  approach  more  extensive- 
ly employs  simulation  runs  than 
does  the  decomposed  approach. 
The  use  of  simulation  runs  in  the 
global  approach  extends  up 
through  certain  contextual  as- 
pects of  performance.  Another 
difference,  of  course,  is  that  the 
global  approach  attempts  to  eval- 
uate military  worth  by  means  of 
global  judgments,  and  the  de- 
composed approach,  in  this  case, 
by  means  of  a formal  MAU 
model. 

The  Navy  preferred  the  latter 
approach  for  basically  two  rea- 
sons. It  found  that  the  level  of 
effort  necessary  to  use  simula- 
tion runs  at  higher  levels  would 
result  in  a cost  exceeding  funds 
allotted  for  evaluation.  And  it 
anticipated  that  the  decomposed 
— henceforth,  MAU — approach 
would  be  more  intelligible  and  its 
results  more  justifiable  than  the 
global  approach.^ 

MAU  Model  Described 

The  essential  fact  about  de- 
composed models,  especially 
MAU  models,  is  that  they  are 
explicitly  structured  to  manage 
information  so  that  one  or  more 
experts  may  contribute  to  the 
final  evaluation  in  a logical  man- 
ner. The  structural  principle  of 
the  MAU  model  used  in  this  case 
was  hierarchical  decomposition ; 
that  is,  each  element  of  the  par- 
ticular evaluation  problem  was 
analyzed  into  its  subelements, 
each  of  which  in  turn  was  ana- 
lyzed into  its  respective  subele- 
ments, and  so  on  until  the  most 
minute  subelements  of  a control- 
lable parameter  of  performance 
demonstrably  affecting  perform- 


ance were  reached.  Accordingly, 
expert  opinions  may  be  selective- 
ly involved  for  the  appropriate 
component  or  subelement.  Con- 
versely, predicted  levels  of  per- 
formance of  the  lowest-level  sub- 
elements in  the  model  may  be 
aggregated  until  a unique  figure 
representing  technical  systems 
utility  is  determined. 

The  hierarchical  structure  of 
the  MAU  model  consisted  of  nine 
levels,  six  of  which  are  illus- 
trated in  Figure  2,  the  remaining 
three  of  which  are  illustrated  in 
Figure  3. 

Level  One  corresponds  to  the 
single  figure  representing  the 
total  worth  of  each  EW  System 
design  proposal. 

Level  Two  represents  three 


major  elements:  military  worth, 
cost,  and  management,  the  first 
of  major  concern  in  this  article. 

Level  Three  represents  the 
conditioning  variables  of  ship 
platforms.  The  five  values  repre- 
sent the  cost-per-unit  ceiling  of 
the  five  EW  suites  constituting 
the  entire  EW  System,  each  cost 
configuration  associated  exclu- 
sively with  a single  class  of  ship 

'The  difference  between  static  and 
dynamic  models  was  not  perceived  to 
affect  the  appropriateness  of  using 
such  a model  for  a dynamic  process. 
Although  MAU  models  are  usually  of 
a static  nature,  they  can  he  used  to 
make  reasonable  approximations  of 
dynamic  processes  under  special  cir- 
cumstances. These  occur  in  the  case 
of  EW  suites  since  their  weapon  de- 
livery times  are  almost  instantaneous. 


40 


October  1977 


platforms.  These  conditioning 
variables  are  necessary  for  de- 
termining the  importance  of 
technical  characteristics  and  ag- 
gregates thereof.  The  MAU 
model  uses  these  variables  in 
distinguishing  the  importance  of 
protecting  one  kind  of  ship 
rather  than  another.® 

Level  Four  represents  eight 
functional  or  deployment  aspects 
of  a system.® 

Level  Five  represents  four 
conditioning  variables,  each  de- 
lineating a different  scenario  de- 
scribing missions  and  threats  to 
the  completion  of  those  missions. 
Although  a scenario  has  no  value 
in  itself,  the  model  indicates  the 
probability  that  the  situations 
described  will  occur  and  the  rela- 
tive importance  of  threats  to  the 
missions. 

Level  Six  represents  four 
kinds  of  responses  to  a threat. 

Level  Seven  represents  the  op- 
erational aspect  of  these  re- 
sponses. 

Level  Eight  and  Level  Nine 


represent  the  parameters  of  per- 
formance associated  with  a par- 
ticular aspect  of  a response  in  a 
particular  situation. 

Note  that  the  heavy  line  in 
Figures  2 and  3 represents  but 
one  possible  thread  through  this 
hierarchy.  Since  there  is  a large 
number  of  possible  threads,  this 
hierarchical  structure  organizes 
in  a manageable  way  the  great 
quantity  of  data  needed  to  deter- 
mine the  worth  of  alternative, 
complex  EW  System  proposals. 

Steps  in  Model  Development 

The  MAU  model  is  developed 
in  five  basic  steps : 

• Decomposing  the  evaluation 
problem  into  its  structured  ele- 
ments. 

• Defining  element  relation- 
ships. 

• Establishing  element  bound- 
aries. 

• Developing  effectiveness 
curves. 

• Determining  element 
weights. 


Although  these  steps  must  be 
listed  and  discussed  sequentially, 
they  should  be  understood  to  be 
taken  iteratively  in  practice,  es- 
pecially the  last  three. 

Decomposing  the  Problem 

The  first  step  is  to  decompose 
the  evaluation  problem  into  its 
structured  elements.  The  prin- 
ciple of  hierarchical  decomposi- 
tion identifies  the  elements  as  it 
structures  them,  as  any  analytic 
process  relates  parts  to  wholes. 
Errors  can,  of  course,  be  cor- 
rected, but  their  rectification  re- 

°As  it  turned  out,  the  evaluation 
revealed  that  the  Navy  maximized 
military  worth  by  developing  an  EW 
system  consisting  of  three,  rather 
than  five,  suites.  It  specified  one  per- 
unit  suite  of  $1.U  million  instead  of 
the  three  suites  with  cost-per-unit 
ceilings  of  $1  million,  $2  million,  and 
$3  million.  That  suite  was  estimated 
to  be  of  greater  worth  than  the  two 
most  expensive  suites  originally  pro- 
posed. 

^During  evaluation,  aircraft  com- 
patibility was  dropped  from  consider- 
ation. 


Figure  2.  The  Structure  of  the  MAU  Model  for  EW  System  Proposals 
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quires  a readjustment  in  all  sub- 
sequent steps  involving  structur- 
ing related  elements. 

There  are  tvi^o  general  kinds 
of  elements  that  emerge  in  devel- 
oping an  MAU  model  of  TSU : 

• The  controllable  parameters 
of  performance,  or  the  technical 
characteristics  of  the  systems. 

• The  factors  mediating  be- 
tween these  parameters  and  the 
aggregate  TSU  of  the  EW  Sys- 
tem. 

All  mediating  factors  define 
the  context  in  which  the  utility 
or  effectiveness  of  each  subfactor 
is  assessed.  One  kind  of  medi- 
ating factor  is  an  aggregate  of 
utility  or  effectiveness  which 
summarizes  measures  of  utility 
or  effectiveness  of  performance 
characteristics.  For  example,  one 


can  speak  of  the  effectiveness  of 
a decoy  as  well  as  of  the  effec- 
tiveness of  its  sensor.  The  other 
kind  of  mediating  factor  is  called 
a conditioning  variable,  a con- 
textual construct  for  modeling 
purposes.  For  example,  although 
one  can  speak  of  the  effective- 
ness of  a decoy,  one  cannot  speak 
of  the  effectiveness  of  an  open 
sea  scenario.  The  distinction  is 
drawn  between  those  factors 
which  pertain  to  the  system 
itself  and  those  which  pertain 
to  constraints  upon  its  cost  or 
to  the  situations  in  which  it  is 
employed. 

Conditioning  variables  consti- 
tute an  explicit  recognition  that 
the  utility  or  effectiveness  of  the 
same  measure  of  performance 
may  change  with  a change  in  the 


context  of  that  performance.  A 
conditioning  variable  is  a set  of 
conditions  that  provides  a con- 
text in  which  measures  of  per- 
formance are  predicted  to  occur 
and  are  evaluated  as  a percent- 
age of  maximum  utility.  In  the 
present  MAU  model  of  TSU, 
there  are  two  levels  of  condition- 
ing variables.  The  first  repre- 
sents four  selected  military  en- 
vironments, called  scenarios,  in 
which  an  EW  suite  may  be  op- 
erationally deployed.  Since  the 
MAU  model  was  used  to  deter- 
mine TSU  of  each  of  five  suites 
defined  by  their  cost  configura- 
tions (cost-per-unit  ceilings), 
the  second  level  of  conditioning 
variables,  the  model  yields  20 
different  measures  of  TSU  for 
each  EW  System  proposal.  These 
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Figure  3.  The  Substructure  of  the  MAU  Model  for  the 
Decoy  in  the  Open  Sea  Scenario 
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measures  permit  discriminations 
of  utility  for  equivalent  per- 
formance. For  example,  at  the 
same  level  of  technical  perform- 
ance, the  utility  of  the  decoy  in 
the  $2  million  suite  on  a cruiser 
in  the  open  sea  scenario  might  be 
far  greater  than  that  in  the 
$500,000  suite  on  a destroyer  in 
the  amphibious  scenario. 

The  use  of  scenarios  in  evalu- 
ation, especially  in  MAU  model- 
ing, is  necessary  as  an  economi- 
cal means  of  predicting  and  eval- 
uating performance,  and  of  re- 
ducing to  a manageable  figure 
the  infinite  number  of  details 
required  to  describe  an  infinite 
number  of  possible  futures. 
Scenarios  serve,  then,  as  classi- 
fying abstractions;  that  is,  sets 
of  generalized  features  which 
characterize  a class  of  possible 
futures.  For  purposes  of  model- 
ing, even  this  greatly  reduced 
number  of  generalized  descrip- 
tions is  neither  economical  nor 
manageable.  Accordingly,  a se- 
lection of  scenarios  must  be 
made  to  ensure  that  the  sam- 
pling is  representative  of  situa- 
tions in  which  the  suites  are 
likely  to  be  employed  on  the  one 
hand  and  that  the  selection  dis- 
criminates among  the  design 
proposals  on  the  other.  The  pro- 
cedures for  selecting  an  econom- 
ical and  manageable  number  of 
scenarios  that  are  at  once  rep- 
resentative and  discriminating 
raise  difficult  methodological 
questions.^  Nevertheless,  the  pro- 
cedures have  been  sufficiently  de- 
veloped and  refined  to  enable  the 
selection  of  scenarios  that  can 
serve  as  a basis  for  dependable 
evaluation.® 

Quantifying  the  Model 

The  remaining  four  steps — 
defining  the  element  relation- 
ships, establishing  element 


boundaries,  developing  effective- 
ness curves,  and  determining  ele- 
ment weights — involve  a highly 
iterative  process.  It  goes  on  con- 
tinuously and  leads  to  refine- 
ments of  initial  quantifications 
until  the  model  is  satisfactory  to 
those  who  must  use  it.  Since  the 
model  must  generate  figures  in- 
dicative of  the  relative  TSU  of 
design  proposals,  the  relation- 
ships among  the  elements  as  well 
as  their  measures  of  utility  and 
effectiveness  must  be  expressed 
mathematically. 

'^The  cost-per-unit  ceil- 
ing and  the  general 
absence  of  perform- 
ance requirements 
make  evaluating  di- 
verse design  proposals 
more  difficult  than  it 
has  been  in  the  past, 
since  all  contractors  are 
likely  to  propose  de- 
signs meeting  the  cost- 
per-unit  ceiling/' 

One  of  the  most  important 
steps  in  developing  the  MAU 
model  involves  a determination 
of  the  rules,  or  formulae,  for 
representing  element  relation- 
ships within  the  model.  The  ele- 
ments subordinate  to  the  same 
immediate  mediating  factor 
combine  to  determine  its  worth 
(a  term  used  to  encompass  either 
utility  or  effectiveness).  The 
worth  of  the  decoy  in  the  open 
sea  scenario  (see  Figure  3)  is 
determined  by  the  aggregated 
worths  of  probability  of  avail- 
ability, sensor  effectiveness,  ef- 
fector effectiveness,  and  system 
reaction  time.  The  rule  for  rep- 
resenting the  relationship  among 
them  must  validly  capture  the 
manner  in  which  they  affect  the 
performance  of  the  decoy  in  the 


open  sea  scenario.  In  short,  the 
combination  rules  define  the 
principles  of  aggregating  the 
worths  of  elements  throughout 
the  MAU  model;  they  capture 
the  value  independence  of  the 
worths  of  the  elements  or  the 
nature  of  their  value  dependency. 
The  relative  simplicity  of  the 
rules  themselves  belies  their 
sophistication;  accordingly,  this 
article  avoids  the  long  and  tech- 
nical discussion  necessary  to  ex- 
plain either  the  rules  or  the  con- 
cepts of  value  dependence  or  in- 
dependence.® 

Once  the  combination  rules  are 
established  (remember  that  they 
are  amended  as  necessary  as  the 
next  two  steps  are  taken),  it  is 
necessary  to  establish  element 
boundaries  on  ranges  of  levels  of 
performance  of  controllable  pa- 
rameters. The  purpose  of  such 
boundaries  is  twofold : they  limit 
the  range  of  performance  to  be 
measured  to  plausible  dimen- 
sions; and  they  permit  the  im- 
portance of  that  performance  to 
be  gauged  realistically.  Too  often 
the  importance  attached  to  a 
specific  kind  of  performance  is 
inflated  because  it  is  considered 
universally,  without  regard  to 
the  range  of  technically  and  eco- 
nomically feasible  levels  of  per- 
formance likely  to  be  suggested 
by  design  proposals.  This  is  one 
point  at  which  the  impact  of  a 

’’The  subject  is  discussed  theoretic- 
ally in  Michael  F.  O’Connor  and  Ward 
Edwards,  ‘’On  Using  Scenarios  in  the 
Evaluation  of  Complex  Alternatives," 
Technical  Report  76—17,  Decisions  and 
Designs,  Inc.,  McLean,  VA,  December 
1976. 

^In  this  case,  the  Navy  reduced  an 
original  selection  of  eight  scenarios  to 
four  and  tested  the  results  obtained 
from  the  MAU  model  by  comparing 
them  with  the  results  of  a much-re- 
duced number  of  simulation  r~uns. 

°See  Hays,  O’Connor,  and  Peterson, 
“Application  of  Multi-Attribute  Util- 
ity Theory,"  pp.  21-25. 


Defense  Management  Journal 


43 


design-to-cost  procurement  pol- 
icy is  felt.  Thus,  whereas  an  ex- 
pert may  wish  the  sensor  of  an 
EW  suite  to  have  a large  capac- 
ity to  detect  discrete  targets  and 
avoid  saturation,  he  may  set  a 
range  of  targets  prior  to  satura- 
tion from  zero  to  25  and  make 
other  judgments  with  respect  to 
that  range,  like  those  for  effec- 
tiveness curves  and  weights. 
However,  if  design  proposals 
drawn  up  in  light  of  cost-per- 
unit  ceiling  are  likely  to  have 
capacities  from  four  to  ten  tar- 
gets prior  to  saturation,  it  is  un- 
likely that  the  expert  will  find 
much  difference  among  the  pro- 
posals in  this  respect  or  attach 
the  proper  degree  of  importance 
to  it.  Accordingly,  experts  must 
set  boundaries  based  on  their 
best  estimates  of  what  range  is 
technically  and  economically 
feasible. 

Completing  the  quantification 
of  the  model  involves  two  more 
iterative  steps : developing  effec- 
tiveness curves  and  determining 
element  weights.  It  is  necessary 
first  to  translate  measures  of 
performance  of  controllable  pa- 
rameters into  levels  of  effective- 
ness expressed  as  percentages, 
then  to  assess  the  importance  of 
the  bounded  ranges  of  perform- 
ance. The  translation  is  achieved 
by  having  the  appropriate  ex- 
perts assess  the  levels  of  effec- 
tiveness (on  a scale  of  zero  to 
100)  for  varying  levels  of  per- 
formance within  previously  de- 
fined boundaries.  The  result  is 
an  effectiveness  curve,  a graphic 
representation  of  the  relation- 
ship between  levels  of  perform- 
ance with  respect  to  a mediating 
factor,  and  effectiveness  (see 
Figure  4).  The  significance  of  an 
effectiveness  curve  is  that  per- 
formance is  immediately  related 
to  and  cannot  be  considered  apart 
from  the  context  in  which  it  oc- 


curs. The  curve  itself  means  that 
performance  is  seen  as  a continu- 
um over  a range  of  levels  of  per- 
formance; a particular  level  of 
performance  is  but  one  possibil- 
ity within  the  range  of  plausible 
levels  of  performance,  not  a 
point  immutable.  Similarly,  ef- 
fectiveness varies  with  the 
range  of  levels  of  performance; 
many  levels  of  performance 
within  the  previously  specified 
boundaries  have  some,  though 
less  than  optimum,  effectiveness 
with  respect  to  the  mediating 
factor.  Effectiveness,  then,  is 
an  intraparametric  measure  of 
worth  throughout  the  plausible 
range  of  levels  of  performance. 


^^One  of  the  most  im- 
portant steps  in  devel- 
oping the  MAU  model 
involves  a determina- 
tion of  the  rules,  or 
formulae,  for  repre- 
senting element  rela- 
tionships within  the 
model" 


The  use  of  effectiveness  curves 
has  two  benefits.  First,  an  ef- 
fectiveness curve  avoids  categor- 
izing a particular  level  of  per- 
formance as  either  acceptable  or 
unacceptable,  as  the  stipulation 
of  system  requirements  de- 
mands. Specific  requirements 
can,  in  fact,  be  represented  by 
effectiveness  curves  in  which  ef- 
fectiveness is  at  a minimum 
when  the  requirement  is  not  sat- 
isfied, at  a maximum  when  it  is. 
Second,  the  particular  shape  of 
an  effectiveness  curve  reveals 
those  ranges  of  performance  in 
which  a slight  change  in  per- 
formance may  result  in  a signifi- 
cant change  in  effectiveness,  for 
better  or  worse,  or  those  ranges 
in  which  a significant  change  in 


performance  may  result  in  little 
or  no  change  in  effectiveness. 
The  effect  of  changing  levels  of 
performance  in  terms  of  TSU 
can  be  readily  displayed,  and  the 
shape  of  an  effectiveness  curve 
m.ay  often  be  diagnostic  of  that 
range  in  which  a trade-off  be- 
tween performance  and  cost  is 
desirable. 

The  final  step  in  this  highly 
iterative  process  in  developing 
an  MAU  model  is  to  determine 
the  relative  importance  to  be  as- 
signed to  subelements  of  the 
same  component.  As  part  of  the 
quantification  of  the  model,  rela- 
tive importance  (determined  by 
a process  called  weighting)  is 
expressed  as  a decimal  between 
0.0  and  1.0.  The  purpose  of 
weighting  is  to  recognize  the 
different  contributions  that  sub- 
element effectiveness  makes  to 
element  effectiveness.  Otherwise, 
the  assumption  is  that  the  effec- 
tiveness of  each  subelement  con- 
tributes equally  to  the  effective- 
ness of  the  element — which  as- 
sumption is  usually  false. 

The  problem  in  weighting, 
then,  is  a methodological  one. 
Since  weights  impinge  upon  the 
figures  for  effectiveness  and  the 
formula  for  aggregating  them, 
the  difficulty  is  twofold : how  to 
assess  weights  and  how  to  in- 
corporate them  into  existing  but 
as  yet  incomplete  combination 
rules  for  aggregating  measures 
of  effectiveness.  Assessing 
weights  depends  upon  the  rela- 
tive influence  a change  in  the 
effectiveness  of  a subelement  has 
upon  the  effectiveness  of  an  ele- 
ment, an  influence  that  may  be 
affected  by  the  relationship 
among  the  subelements.  That  is, 
weights  assigned  to  subelements 
are  a function  of  their  effect 
upon  the  element  as  the  effective- 
ness of  each  changes  from  mini- 
mum to  maximum.  Since  the 
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assessment  of  weights  depends 
upon  the  nature  of  the  combina- 
tion rules,  this  technical  discus- 
sion is  also  avoided. 

Validating  the  Model 

Once  the  MAU  model  is  devel- 
oped, two  types  of  validation 
analyses  are  conducted.  The  first, 
internal  validation,  consists  of 
examining  the  effects  of  varia- 
tions in  weights  and  effective- 
ness curves  for  purposes  of  iden- 
tifying critical  aspects  of  the 
model.  This  process,  known  as  a 
sensitivity  analysis,  has  shown 
that  MAU  models  are  usually  de- 
pendable despite  small  errors  in 
utility  functions  or  weights.  It  is 
also  typical  that  MAU  models  are 
sensitive  to  structural  changes 
resulting  from  different  inter- 
relationships among  model  fac- 
tors or  to  associated  changes  in 
combination  rules.  It  is  the  struc- 
ture of  the  model  that  either  does 
or  does  not  capture  the  true  na- 
ture of  the  problem.  Internal 
sensitivity  analyses  cannot,  how- 
ever, guarantee  this  type  of 
validity. 

The  second  type  of  validation, 
external  validation,  provides  evi- 
dence of  the  validity  of  the  model 
structure  by  using  known  sys- 
tems to  calibrate  the  model.  The 
model  should  assign  to  these 
known  systems,  whose  record  of 
performance  has  been  well  doc- 
umented, values  that  are  consist- 
ent with  the  generally  accepted 
assessments  of  the  systems  based 
upon  that  record.  If  it  does  so, 
it  helps  to  ensure  that  the  model 
behaves  as  it  should.  If  not, 
structural  changes  are  required. “ 

“ Ibid.,  pp.  27-29. 

In  the  actual  external  validation 
of  the  MAU  model,  it  was  found  that 
not  enough  credit  had  been  given  the 
sensor  for  its  ability  to  detect  false 
alarms.  The  model  was  altered  to  ac- 
count for  this  fact. 


Figure  4.  Effectiveness  Curve  for  Detection  by  the  Sensor 
of  the  Decoy  in  the  Dpen  Sea  Scenario 
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Figure  5.  Effectiveness  of  the  Sensor  of  the 
Decoy  in  the  Open  Sea  Scenario 
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The  implementation  of  such 
models  on  interactive  computers 
also  allows  “what-if”  type  analy- 
ses that  can  accommodate  pro- 
posed design  changes.  Similarly, 
the  model  can  actually  be  used  to 
search  for  those  feasible  config- 
urations that  have  maximum 
utility. 

An  example  of  what  the  com- 
pleted model  is  like  is  suggested 
by  that  part  of  it  for  the  effec- 
tiveness of  the  sensor  of  the 
decoy  in  the  open  sea  scenario,  as 
shown  in  Figure  5.  Effectiveness 
curves  would  assess  the  percent- 
ages of  effectiveness  for  the  esti- 
mated level  of  performance  of 
smallest  parameters.  These  per- 
centages would  be  multiplied  by 
their  weights,  and  related  prod- 
ucts would  be  combined  accord- 
ing to  the  rule  implied  by  the 
signs  (here,  by  addition  in  all 
cases).  The  resulting  figures  for 
the  effectiveness  of  the  subele- 
ments or  parameters  relating 
them  are,  in  turn,  multiplied  by 
their  weights,  and,  again,  the 
products  are  combined  according 
to  the  rule  implied  by  the  signs. 
Note  in  this  instance  that  the 
figures  for  three  subelements 
and  an  additive  constant  are 
summed  before  multiplication  by 
the  subelement  for  detection. 
(The  additive  constant  in  this 
case  says  that  the  sensor  has 
some,  potentially  70  percent,  ef- 
fectiveness if  it  merely  detects 
a target.)  By  a repetition  of 
these  steps  throughout  and  up 
the  model,  figures  are  aggregated 
until  they  yield  a single  figure 
representing  the  military  worth 
of  an  EW  System  design  pro- 

^The  MAU  model  was  implemented 
on  computers  at  the  Naval  Research 
Laboratories,  Washington,  DC,  and 
was  used  both  for  numerous  “what-if" 
type  analyses  in  the  procurement 
stages  as  well  as  for  other  design 
problems  on  related  projects. 


posal.  Theoretically,  if  an  MAU 
model  completely  encompasses 
those  considerations  entering  in- 
to a particular  stage  of  the  pro- 
curement cycle,  a ranking  of  the 
figures  should  select  those  pro- 
posals for  engineering  develop- 
ment for  prototype  testing. 

Ultimate  Benefit 

The  use  of  MAU  models  is  not 
likely  to  be  confined  solely  to  the 
evaluation  and  selection  of  de- 
sign proposals  and  prototypes  in 
the  procurement  cycle.  It  should 
be  clear  that  an  MAU  model  can 
also  be  used  to  design  a system 
and  diagnose  areas  in  which 
technological  development  is 
worthwhile  or  not.  It  is  quite 
likely,  therefore,  that  MAU 
models  will  be  employed  by  con- 
tractors to  assist  in  the  design 
of  other  sophisticated  and  com- 
plex major  weapon  systems. 
When  the  use  of  MAU  models 
becomes  commonplace,  the  mili- 
tary services  and  contractors 
will  be  better  able  to  discuss  the 
performance  of  major  military 
systems  in  a systematic  manner. 
For  one  of  the  most  significant 
advantages  of  MAU  models  is 
their  potential  for  promoting 
effective  communication.  DJV|J 
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Conflict  In  Organizations: 


Good 

or 

Bad 


The  extent  to  which  conflict  is  a positive  or  negative  factor  in 
reaching  the  objective  is  largely  determined  by  the  manager's 
perspective. 


tT  e live  in  an  age  saturated  with  many  forms 
and  intensities  of  conflict.  The  term  conflict  is 
used  to  describe  important  differences  between 
individual  human  beings  or  groups  of  human 
beings.  Conflict  is  a major  organizational  real- 
ity. Typical  of  this  conflict  is  the  breakdown 
of  communication  that  can  occur  between  man- 
agement and  labor,  between  a commander  and 
his  troops,  or  between  a subordinate  and  a 
supervisor. 

In  our  age  of  dynamic  change  it  is  imperative 


that  the  manager  understand  some  of  the  basic 
causes  of,  and  various  strategies  for  dealing 
with,  the  conflict  which  inevitably  occurs  in  or- 
ganizations. Not  all  conflict  is  bad.  Differ- 
ences which  result  in  initiative  and  creativity 
are  stimulating  for  those  involved,  and  such 
conflict  is  essential  for  progress. 

Nations,  organizations,  and  groups  are  made 
up  of  individual  human  beings.  Through  life 
experiences,  each  human  being  has  developed 
a set  of  values  and  evolved  a set  of  behavioral 
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rules.  These  values  and  rules  are  sufficiently 
alike  in  a given  society  to  allow  justice,  morals, 
and  ethics  to  exist  and  create  general  agree- 
ment about  what  is  right  and  what  is  wrong. 
But  the  value-rule  set  for  each  individual  is  a 
unique  set  not  fully  shared  by  others.  These  dif- 
ferences in  value-rule  sets  are,  most  likely,  the 
basic  causes  of  conflict.  An  airman,  for  instance, 
might  be  incensed  over  what  he  considers  a 
wrong  since  the  base  commander  has  denied  him 
and  his  family  base  exchange  check-cashing 
privileges  because  a series  of  bad  checks  re- 
sulted from  a pay  record  error.  However,  the 
base  commander,  bothered  by  numerous  bad 
check  experiences  through  the  BX,  may  feel 
it  wrong  not  to  punish  such  occurrences. 

Another  cause  of  conflict  is  the  motivation 
of  separate  individuals.  Each  is  motivated  by  a 
peculiarly  unique  degree  of  satisfactions  in  a 
set  of  needs.  It  is  quite  likely  that  in  a given 
group  situation  the  individuals  concerned  will 
be  aiming  their  personal  efforts  at  slightly  dif- 
ferent objectives ; such  objectives  may  be  simi- 
lar enough  to  permit  cooperative  effort  but 
sufficiently  different  to  create  some  conflict. 
A common  example  might  be  the  hot  line,  em- 
ployed by  many  commanders  as  one  of  several 
means  of  staying  in  touch  with  the  troops.  This 
opportunity  to  short-circuit  supervisory  chan- 
nels often  antagonizes  intermediate  managers, 
who  may  learn  of  a problem  only  when  the 
commander  confronts  them  with  it.  Then,  too, 
it  is  possible  for  all  to  be  motivated  to  behave 
toward  the  same  goal  accomplishment  but  to 
feel  that  that  goal,  when  attained,  will  not  be 
great  enough  for  all  to  share  adequately  in  the 
reward.  Conflict  may  then  exist  as  each  strives 
to  attain  his  place  in  the  sun. 

A third  major,  and  more  obvious,  cause  of 
conflict  is  found  in  the  differing  ideologic  and 
philosophic  bases  we  possess.  These  relate  to  a 
great  extent  to  the  value-rule  set  but  are  suffi- 
ciently different  to  warrant  recognition  as 
possible  conflict  causes.  What  we  use  as  a base 
for  our  ideals  and  our  concepts  becomes  of  great 
importance  to  us,  and  we  do  not  want  that  base 
challenged  or  questioned  by  others.  An  example 
is  the  old-timer’s  reluctance  to  change  from  a 
thus-far  successful  technique  even  though  evi- 
dence indicates  that  a change  would  be  bene- 
ficial. When  we  perceive  attack  on  our  ideals 
from  another,  we  respond  energetically  and 


often  heatedly.  Note  the  actions  of  some  militant 
minority  groups  and  you  observe  ideologic/ 
philosophic  conflict  relations  at  work. 

Effects  of  Conflict 

Conflict  has  both  positive  and  negative  ef- 
fects. It  can  be  positive  when  it  encourages 
creativity,  new  looks  at  old  conditions,  the  clari- 
fication of  points  of  view,  and  the  development 
of  human  capabilities  to  handle  interpersonal 
differences.  All  of  us  have  experienced  a surge 
of  creativity  when  we  permit  the  ideas  of  others 
to  trigger  our  imagination,  as  for  example  in  a 
brainstorming  session.  Conflict  can  be  negative 
when  it  creates  resistance  to  change,  establishes 
turmoil  in  organization  or  interpersonal  rela- 
tions, fosters  distrust,  builds  a feeling  of  de- 
feat, or  widens  the  chasm  of  misunderstanding. 
Such  might  be  the  situation  today  in  American 
society  relative  to  school  busing. 

Unfortunately,  the  term  “conflict”  has  only 
the  connotation  “bad”  for  many  people,  so  much 
so  that  they  think  principally  in  terms  of  sup- 
pression, giving  little  or  no  attention  to  its 
more  positive  side.  One  author  emphasizes  this 
by  stating : “It  seems  entirely  likely  that  many, 
if  not  most,  organizations  need  more  conflict, 
not  less.”  ^ Another  states : “The  absence  of 
conflict  may  indicate  autocracy,  uniformity, 
stagnation,  and  mental  fixity;  the  presence  of 
conflict  may  be  indicative  of  democracy,  diver- 
sity, growth,  and  self-actualization.”  ^ Some 
social  critics  relate  the  military  to  the  former 
by  their  references  to  the  military  mind  in 
which  they  equate  absolute  and  unquestioning 
obedience  with  normal  military  functioning. 

Conflict  should  be  considered,  conceptually, 
as  neither  bad  nor  good,  wrong  nor  right.  The 
meaning  of  conflict  is  established  by  its  par- 
ticipants, since  it  is  people  who  attach  value 
definition  to  it.  The  ultimate  results  of  a con- 
flict situation  are  determined  by  the  feelings, 
beliefs,  and  values  of  the  persons  involved. 
People  are  the  real  determinants  of  the  meaning 
of  conflict.  If  we  forget  this  and  treat  conflict 
as  though  it  had  some  natural  quality  (good/ 
bad,  right/wrong),  we  overlook  the  roles  of  the 

^Stephen  P.  Robbins,  Managing  Organizational  Con- 
flict (Englewood  Cliffs,  NJ : Prentice-Hall,  197i),  p. 
19. 

^Leonard  Rico,  “Organizational  Conflict:  A Frame- 
work for  Reappraisal,”  Industrial  Management  Re- 
view, Fall  1964,  p.  67. 
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participants  and  probably  lose  the  ultimate 
capability  of  stimulating  conflict. 

We  are  human,  though,  and  it  is  almost  im- 
possible for  us  to  divorce  ourselves  from  feel- 
ings, beliefs,  and  values.  We  create  or  get  in- 
volved in  conflict,  and  we  possess  predisposi- 
tions as  to  how  it  ought  to  be  addressed  or 
handled.  We  tend  to  have  a strong  behavioral 
leaning,  a set  pattern,  for  our  participation, 
and  this  emerges  as  a major  factor  in  setting 
the  nature  of  conflict.  We  can  note  this  pre- 
disposition for  a set  pattern  of  behavior  in  our 
tendency  to  want  to  apply  equal  penalties  or 
identical  punishment  regardless  of  the  cause  of 
an  infraction  of  rules. 

We  can  say,  then,  that  conflict  is  a state  of 
unresolved  difference  between  two  individuals, 
an  individual  and  a group,  or  two  groups.  The 
difference  can  be  real  or  imaginary.  Regardless, 
it  is  a difference  and  will  cause  some  form  of 
conflict  if  the  involved  parties  are  in  contact 
with  each  other.  The  conflict  exists  until  the 
difference  is  resolved.  The  important  aspect  is 
how  the  individual  accepts  and  responds  to  it, 
how  he  seeks  to  control  or  stimulate  the  dy- 
namic conflict  situation. 

In  this  age  of  specialization  and  sophisticated 
technology,  we  can  readily  find  power  imbal- 
ances in  organizations.  This  often  results  in 


conflict.  In  technically  oriented  organizations, 
the  managers  rarely  are  able  to  be  experts  in 
all  the  disciplines  or  specialties  they  control. 
These  managers  find  themselves  greatly  depend- 
ent on  technical  experts  who  work  for  them. 
Differences  arise  because  of  differing  knowledge 
bases  and  perceptions.  Note  the  hard  feelings 
and  accompanying  resentment  we  often  expe- 
rience when  a boss,  removed  from  the  situation, 
nitpicks  a piece  of  correspondence  we  have 
prepared.  Unmanaged,  these  differences  can 
have  negative  results.  But  this  need  not  be  if 
we  carefully  select  mature  and  adaptable  man- 
agers who  can  understand  the  high  degree  of 
informal,  expert  authority  held  by  subordinates 
who  have  technical  competence. 

The  Need  to  Manage  Conflict 

We  must  expect  conflict  to  occur  in  our  orga- 
nizations. We  should  be  disappointed  if  it  does 
not  because  conflict  exists  only  within  the 
context  of  interdependence.  There  can  be  no 
conflict  when  there  is  no  awareness  of  another 
meaning,  role,  or  value  than  our  own.  Thus, 
conflict  is  a relationship  between  segments  of 
an  interrelated  system : persons,  a group,  an 
organization,  a community,  a nation.  There 
can  be  no  conflict  if  those  involved  sense  no 
differences.  However,  in  the  environment  of 
interpersonal  relationship  there  will  always  be 
differences,  and  conflict  will  be  the  norm,  not 
the  exception. 

We  need  to  manage  conflict  in  order  to  obtain 
profitable  return  from  it.  Managing  conflict 
requires  that  we  consider  not  only  the  guidance 
and  control  required  to  keep  it  at  an  acceptable, 
yet  not  too  high,  level,  but  also  the  activity 
necessary  to  encourage  proper  conflict  when  the 
level  is  too  low.  Who  would  want  to  lead  an 
organization  without  the  energy  and  force 
accompanying  the  conflict  of  creativity  and 
initiative? 

Stephen  Robbins  makes  a strong  case  for 
the  need  for  a more  realistic  view  of  conflict 
with  his  interactionist  approach.®  He  states  that 
there  are  three  basic  managerial  attitudes  to- 
ward conflict  which  he  identifies  as  traditional, 
behavioral,  and  interactionist.  The  traditional- 
ist believes  that  all  conflicts  are  destructive  and 
that  management’s  role  is  to  get  them  out  of 
the  organization.  The  traditionalist,  therefore, 

^Robbins,  pp.  11-21. 
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believes  conflict  should  be  eliminated.  The  be- 
havioralist  seeks  to  rationalize  the  existence  of 
conflict  and  accurately  perceives  conflict  as  in- 
evitable in  complex  organizations  or  relation- 
ships. Thus,  the  behavioralist  “accepts”  it.  The 
interactionist  views  conflict  as  absolutely  neces- 
sary, encourages  opposition,  defines  manage- 
ment of  conflict  to  include  stimulation  as  well 
as  resolution,  and  considers  the  management  of 
conflict  as  a major  responsibility  of  all  admin- 
istrators. The  interactionist  view  is  to  accept 
and  encourage  conflict.  This  article  uses  the 
interactionist  approach. 

The  Person  and  the  Organization 

Conflicts  occur  when  the  needs  and  goals  of 
the  individuals  are  not  in  harmony  with  the 
needs  and  goals  of  the  organization.  Chris 
Argyris,  in  his  discussion  of  man  versus  the 
organization,  indicates  it  is  highly  conceivable 
that  the  traditional  goals  and  structures  of  or- 
ganizations may  be  in  conflict  with  the  needs 
and  goals  of  a developing  personality.^  This 
may  be  readily  seen  in  the  efficient  and  omnipo- 
tent bureaucracy  that  places  emphasis  on  hier- 
archy, specialization  of  work,  established  norms 
of  conduct,  and  explicit  rules,  often  forgetting 
or  overlooking  the  individual  and  his  unique 
qualities.  Traditionally,  personal  values  tend  to 
be  hostile  toward  organizations,  big  govern- 
ment, big  business,  bureaucracy,  and,  in  spite 
of  its  purely  defensive  posture  in  our  country, 
the  military. 

Again,  conflict  can  arise  when  interdepend- 
ency exists.  Employees  become  dependent  on 
organizations  to  give  their  lives  direction  and 
meaning.  Such  dependency  allows  them  to  es- 
cape the  burdens  of  personal  responsibility. 
Whereas  we  praise  individualism  in  workers, 
the  organization  often  requires  that  the  indi- 
vidual be  treated  impersonally.  We  see  this  in 
“distant”  management,  in  which  the  people 
sense  an  absence  of  concern  for  their  individ- 
uality and  personal  needs. 

Efficiency  requirements  of  the  organization 
also  act  as  sources  of  conflict  because  they  regu- 
larly demand  that  the  goals  and  needs  of  the 
organization  be  given  higher  priority  than  the 
rights  of  the  individual. 

Managing  this  inherent  conflict  between  in- 

'Chris  Argyris,  Personality  and  Organization  (New 
York:  Harper  Torchbooks,  1957),  Chapters  III  and 
VII. 


dividual  and  organizational  needs  demands  a 
high  degree  of  self-awareness  on  my  part  as 
a manager.  What  am  I willing  to  do  in  balanc- 
ing these  needs?  How  much  can  I accommodate 
comfortably  to  the  need  satisfaction  of  other 
individuals  in  my  organization?  How  much 
faith  do  I really  have  in  the  motivational  drives 
of  my  subordinates?  What  really  is  my  role  in 
this  unit?  What  can  I do,  or  what  will  I allow 
myself  to  do,  to  integrate  the  needs  of  the 
individual  with  the  needs  of  the  organization? 
In  this  circumstance  that  now  faces  me,  which 
is  more  important:  the  individual  or  the  group? 
No  one  can  preanswer  these  questions,  nor  can 
anyone  answer  them  effectively  for  another  per- 
son. Yet  the  active  manager  has  to  answer 
them  as  he  strives  to  control  conflict. 

Never  is  the  functioning  organization  free  of 
problems.  The  unresolved  problem  is  a source 
of  conflict  because  individuals  are  expected  to 
solve  the  problem,  but  the  organization  (man- 
agement?) often  does  not  permit  mistake  or 
error,  or  the  organization  often  gives  the  indi- 
vidual a problem  so  huge  it  overwhelms  him. 
Frustration  and  conflict  naturally  result.  Un- 
less the  organization  is  supportive  to  the  indi- 
vidual’s problem-solving  efforts,  such  conflict 
continues  and  likely  worsens.  What  is  needed, 
as  Harry  Levinson  indicates,  is  a supportive 
environment  that  gives  the  individual  room  to 
maneuver  and  freedom  to  make  mistakes,  set 
limits,  and  define  expectations,  plus  respectful 
treatment  of  his  ideas.® 

In  many  of  today’s  organizations,  such  a 
supportive  environment  is  contrary  to  devel- 
oped functional  relationship  patterns.  In  far 
too  many  organizations,  for  example,  mistakes 
are  anathema,  and  more  effort  is  expended  in 
protective  posterior  armor  than  in  productive 
and  progressive  activity.  Fear  is  prevalent,  and 
the  feeling  of  individual  versus  the  organization 
is  magnified.  Many  people  in  military  organiza- 
tions experience  this  as  they  find  they  must 
guard  against  inspection  visits,  staff  visits,  and 
the  like,  at  the  expense  of  a ready  solution  to  an 
immediate  problem.  In  some  instances  they  fol- 
low the  book,  even  knowing  it  to  be  in  error  in 
a given  situation,  because  they  cannot  antici- 
pate support  for  an  innovative  action. 

An  additional  source  of  individual  versus  or- 
^Harry  Levinson,  The  Exceptional  Executive:  A 

Psychological  Conception  ( Cambridge,  MA : Harvard 
University  Press,  1968),  pp.  20Jt-7. 
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ganization  conflict  is  generated  by  the  atmos- 
phere of  the  “new  man”  versus  the  “old  man.” 
The  innovator  is  always  in  a less  supportive 
environment  than  the  entrenched  old  hand. 
Interpersonal  conflicts  of  the  old  and  new  vary 
in  intensity  in  relation  to  the  ability  of  the 
manager  to  deal  with  such  conflicts  and  his 
personal  desire  to  benefit  from  innovative  ideas. 
But  the  new  is  not  always  right,  and  we  must 
rationally  evaluate  these  old  versus  new  argu- 
ments. Frederick  Herzberg  speaks  to  this  point 
in  his  recommendations  for  management’s  re- 
ferral use  of  “the  wise  old  Turk,”  ® a valuable 
source  of  information  already  on  the  payroll. 

Reactions  to  Conflict 

Since  conflict  may  be  positively  or  negatively 
evaluated,  there  may  be  a range  of  reaction  to 
it.  These  reactions  might  go  from  high  expec- 
tation and  pleasure  to  absolute  rejection.  In  a 
very  broad  sense,  the  individual  in  a conflict 
situation  has  only  two  options : sign  up  or  ship 
out. 

Rejection  of  the  conflict  situation  may  result 

^Frederick  W.  Herzberg,  “The  Wise  Old  Turk,” 
Harvard  Business  Review,  September -October  197U, 
pp.  70-80. 


in  shipping  out,  a resignation  which  may  be 
temporary  or  permanent.  The  response  might 
be  as  mild  as  taking  a few  days  of  respite,  thus 
the  therapeutic  value  of  leave,  vacation,  and 
recreation.  Perhaps,  in  certain  organizations, 
it  would  be  a sabbatical  or  volunteering  for 
special  duty  in  a new  environment.  Then,  too, 
it  might  be  total  severance,  with  the  goal  of  a 
fresh  start  in  a different  organization.  Or,  it 
might  be  using  the  personnel  system  to  find  a 
clean  start  through  internal  transfer  to  another 
subelement  of  the  organization. 

Acceptance  of  the  conflict  situation  might  be 
manifested  in  a surge  of  initiative,  a flow  of 
creativity,  or  a push  for  productivity.  These 
efforts  might  result  from  stimulation  of  per- 
ceived differences,  or  they  might  be  the  observ- 
able behavior  representing  a strong  desire  for 
promotion  and,  thus,  escape  from  the  conflict. 
The  net  effect  may  well  be  good  for  both  the 
organization  and  the  person. 

Means  to  Resolve  or  Reduce  Conflict 

Basic  to  other  considerations  in  dealing  with 
conflict,  conflict  resolution  requires  that  the 
parties  in  conflict  trust  each  other  and  that  they 
are  able  and  willing  to  locate  the  source  of  the 
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conflict.  Then  too,  since  a man  convinced 
against  his  will  is  not  convinced,^  we  can  gen- 
erally eliminate  the  archaic,  although  often- 
used,  hammer  on  the  head  method.  Putting  the 
lid  on  conflict  does  nothing  about  eliminating 
its  source. 

We  might,  in  a conflict  situation,  do  nothing 
about  it.  What  would  be  the  result  if  we  de- 
cided to  take  no  action  to  deal  with  conflict 
that  has  been  discovered  to  be  bad  for  the  or- 
ganization? If  an  individual  or  group  remains 
in  conflict,  there  will  be  increased  tension  that 
sooner  or  later  will  result  in  one  striving  to 
win  and  drive  the  loser  out  of  the  situation. 
Even  worse,  the  losing  element  may  become 
increasingly  more  aggressive  or  hostile  and 
counterattack  the  element  frustrating  it.  At 
any  rate,  the  result  is  likely  to  be  dysfunctional. 
So,  as  a normal  thing,  the  decision  to  do  nothing 
is  probably  not  the  best.  However,  the  manager 
on  the  scene  must  make  this  determination.  He 
must  understand  that  there  are  times  when 
the  decision  to  do  nothing  may  be  best.  This 
decision  can  only  be  made  by  someone  in  the 
situation  evaluating  the  forces  and  strengths 
involved. 

A frequently  used  method  for  resolving  con- 
flict is  the  use  of  superordinate  goals.  For  ex- 
ample, the  entire  work  force,  taken  as  a whole, 
is  something  of  a superordinate  goal  uniting 
conflicting  groups  beneath  the  umbrella  of  that 
goal.  The  manager  gets  the  groups  to  see  how 
the  conflict  serves  to  reduce  productivity,  there- 
by reducing  the  smaller  group’s  stake  in  the 
benefits  of  the  major  organization’s  success. 
Even  though  the  source  of  conflict  is  not  treated, 
it  is  an  important  first  step  because  it  sets  the 
stage  for  compromise.  This  approach  is  similar 
to  the  common  enemy  approach,  wherein  groups 
in  competition  find  unity  viewing  an  outside 
group  as  a common  enemy.  This  unity  can  hide, 
or  make  less  important,  conflicts  within  the 
group. 

A unique  method  to  resolve  conflict  is  to 
increase  interaction  between  conflicting  groups 
by  physically  exchanging  persons  between  con- 
flicting groups.  For  example,  if  the  gizmo  unit 
is  having  difficulty  dealing  with  the  gadget  unit, 
a temporary  shifting  of  people  between  these 

’’For  an  interesting  account  see  Captain  Paul  B.  Ryan, 
USN  (Retired),  "USS  Constellation  Flare-up:  Was  It 
Mutiny?"  United  States  Naval  Institute  Proceedings, 
January  1976,  pp.  1)6-53. 


groups  could  help  the  conflicting  elements  learn 
the  other’s  problems  and  frames  of  reference. 
The  result  should  be  better  communications, 
greater  understanding,  and  less  future  conflict. 

The  quickest  resolution  is  a confrontation 
meeting.  The  manager  should  be  warned,  how- 
ever, that  confrontation  requires  complete  pre- 
paredness on  his  part.  He  must  have  the  facts 
of  the  conflict  situation  and  confidence  in  his 
self-control  and  his  ability  to  use  diplomacy, 
tact,  and  problem  solving.  Even  then,  he  must 
also  accept  the  possibility  that  a confrontation 
may  worsen  the  situation.  Basic  to  his  efforts 
to  resolve  or  reduce  the  conflict  must  be  the 
idea  of  avoiding  win-lose  situations.  Sports  and 
other  recreational  activities  often  acquire  their 
flavor  by  win-lose  situations,  but  the  same  win- 
lose  options  are  not  always  desirable  in  orga- 
nizational functions.  Far  too  often  in  organi- 
zations, this  results  in  suboptimization.  A sub- 
element may  become  so  involved  in  winning 
that  it  loses  sight  of  the  overall  mission  of  the 
larger  unit  it  serves,  and  its  efforts  become 
counterproductive. 

Most  complex  organizations  have  reward  sys- 
tems based  upon  collaborative  effort.  The  orga- 
nization that  depends  upon  coordinative,  co- 
operative work  may  be  mortally  wounded  if  its 
subelements  acquire  win-lose  attitudes  which 
cause  these  suboptimizing  activities.  Once  the 
stage  is  set  by  the  manager,  he  may  initiate 
negotiation  by  representatives  of  the  conflicting 
groups.  During  this  negotiation,  the  manager 
may  wish  to  use  an  impartial  judge  or  arbi- 
trator to  listen  to  arguments  from  both  sides 
and  seek  to  find  points  of  possible  agreement  or 
compromise.  Of  course,  we  recognize  this  as 
the  usual  last  resort  in  management-labor  diffi- 
culties and  severe  conflict  situations. 

There  are  situations  in  which  the  manager 
must  seek  to  repress  conflict.  This  is  especially 
true  when  the  differences  between  the  con- 
flicting elements  are  not  relevant  to  the  orga- 
nizational task.  This  occurs  when  two  partici- 
pating people  have  off-the-job  differences  which 
they  permit  to  enter  the  world  of  work.  Nor- 
mally this  type  of  conflict  is  bad  for  the  orga- 
nization. Often  these  differences  are  petty  and 
self-serving,  thereby  causing  activity  in  which 
the  participants  try  to  win  to  preserve  the 
sanctity  of  their  original  stand.  A significant 
aid  to  the  manager  in  this  form  of  conflict  is 
a well-developed  understanding  of  the  human 
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process  of  perception,  the  process  by  which  we 
handle  stimuli  in  accordance  with  our  values, 
rules,  wishes,  and  fears.  With  this  understand- 
ing, the  manager  might  be  able  to  explain  to  the 
conflicting  parties  how  they  are  misreading  the 
situational  data.  He  might  then  obtain  agree- 
ment of  a sort  that  causes  the  conflict  to  be 
repressed. 

All  conflict  is  not  bad.  There  will  be  times 
when  a manager  wants  conflict  of  the  right 
type ; therefore,  it  would  be  advantageous  for 
him  to  know  some  means  to  stimulate  it.  In  a 
number  of  instances,  he  might  strive  to  create 
the  situations  he  earlier  worked  to  eliminate. 
For  example,  he  might  create  win-lose  situa- 
tions in  which  a form  of  competitiveness  might 
be  engendered.  This  often  works  in  such  areas 
as  selling  an  idea,  recognizing  the  creation  of 
new  approaches  to  organizational  success,  and 
so  forth.  One  means  to  do  this  is  to  deemphasize 
the  need  for  everyone  to  contribute  to  overall 
organizational  success.  That  is,  the  manager 
begins  to  emphasize  the  accomplishments  or 
performance  of  individual  people  or  separate 
units  in  lieu  of  stressing  the  performance  of  the 
whole.  He  must  be  cautious,  though,  to  avoid 
creating  a monster  that  becomes  an  even 
greater  problem  than  the  absence  of  productive 
conflict. 


Individuals  are  the  creative  segments  of  so- 
ciety. True,  the  synergism  of  two  or  more  indi- 
viduals often  makes  us  think  of  organizational 
creativity,  but  it  really  is  the  individual  who 
creates.  Therefore,  stimulation  of  creative  con- 
flict can  be  obtained  by  increasing  the  autonomy 
of  individuals  on  their  jobs.  A less  demanding 
imposed  structure,  granting  more  freedom  for 
the  individual  to  choose  and  decide  for  himself, 
usually  creates  an  environment  in  which  the 
creative  nature  is  fanned  to  flame.  Similarly,  a 
decrease  in  supervisory  overhead  (a  widening 
of  the  organization)  can  accomplish  this  result. 
Again,  though,  the  manager  must  remain  in 
control  of  the  situation  lest  it  get  out  of  hand. 
It  is  sometimes  easy  to  forget  the  real  goals 
of  the  organization  as  we  get  enmeshed  in  the 
thrill  of  innovation. 

Conflict  is  a state  of  unresolved  difference 
between  two  entities,  human  or  organizational. 
Sometimes  the  difference  is  functionally  pro- 
ductive, as  with  creativity;  but  sometimes  it  is 
dysfunctional,  as  with  war  or  sabotage  or  other 
less  drastic  confrontations.  Conflict  should  not, 
therefore,  be  considered  naturally  bad  or  good. 
It  will  be  bad  or  good  depending  upon  the  value 
base  of  the  interpreter.  But  conflict  of  some 
form  is  inevitable  whenever  two  or  more  human 
beings  are  in  some  interdependent  relationship. 
The  important  aspect  of  conflict  is  how  the 
human  participants  relate  and  respond  to  it. 
Managers  must  control  conflict.  That  is,  they 
must  keep  dysfunctional  conflict  at  an  accept- 
able level,  but  they  must  also  learn  to  stimulate 
functionally  productive  conflict  when  it  is  at 
too  low  a level.  DJV|J 
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Reaching  A Supply  Efficiency  Peak 
Through  Selective  Management 


Bij  investing  more  heavihj  in  its  faster-moving  stock,  the  Defense 
Electronics  Supply  Center  achieved  the  best  efficiency  record 
in  its  history  during  a recent  two-year  period. 


\y/  hat  does  an  electronics  item 
have  in  common  with  a pair  of 
overalls?  Not  much.  But  draw- 
ing on  a simple  and  logical  busi- 
ness principle  used  in  selling 
overalls  helped  the  Defense  Elec- 
tronics Supply  Center  in  Dayton, 
Ohio,  achieve  the  best  supply 
efficiency  record  in  its  history 
during  a recent  two-year  period. 

Like  the  other  five  supply  cen- 
ters under  the  Defense  Logistics 
Agency,  DESC  operates  on  a re- 
volving stock  fund  basis,  where- 
by materials  are  “sold”  to  mili- 
tary and  civil  agency  customers 
at  cost  plus  a surcharge  to  cover 
operating  overhead. 

Since  new  stock  is  purchased 
with  sales  revenue,  too  much 
money  invested  in  slow-moving 
items  reduces  the  ability  to  re- 
plenish fast-moving  stock.  By 
the  same  token,  extra  money 


by  Stephen  A.  Stromp,  Jr. 
Public  Affairs  Office 
Defense  Electronics  Supply  Center 
Dayton,  OH 


put  into  fast  movers  increases 
the  total  stock  availability. 

With  this  in  mind,  DESC 
adopted  the  philosophy  of  “buy- 
ing most  of  what  sells  best.”  As 
a result,  the  Center’s  stock  avail- 
ability rate,  that  percentage  of 
requisitions  that  could  be  com- 
pletely shipped  upon  initial  re- 
view by  DESC,  rose  substan- 
tially during  fiscal  years  1975 
and  1976  while  the  volume  of 
backorders  declined  markedly. 

The  Management  Approach 

The  system  employed  by 
DESC  to  manage  its  multimil- 
lion-dollar  inventory  of  over  half 
a million  electronic  items  is  es- 
sentially the  same  as  approaches 
used  by  most  supply  activities, 
but  the  Center  is  distinctive  in 
its  style  of  management  and  the 
degree  of  intensity  with  which 
it  is  applied.  Rather  than  buying 
“more”  of  what  sells  best,  DESC 
buys  “most.” 

The  concept  is  patterned  after 
proven  retailing  methods  which 
apportion  investment  relative  to 


sales.  As  related  by  Robert  L. 
McBride,  deputy  director  of  sup- 
ply operations  at  DESC,  “I  was 
first  introduced  to  the  method 
years  ago  while  working  part- 
time  in  a department  store.  I 
noticed  we  were  never  out  of 
stock  of  size  32/30  overalls. 
Being  a rural  community,  jeans 
were  a hot  seller,  and  on  Friday 
evenings  we  always  stocked  the 
shelves  with  32/30s  in  anticipa- 
tion of  Saturday’s  business.  But 
52/28s  were  another  story.  We 
may  have  carried  one,  but  never 
more  than  three  because  of  very 
minimal  demand.” 

Through  this  approach  of 
placing  money  on  the  movers 
while  also  keeping  enough  low- 
demand  stock  on  hand  to  assure 
proper  support  to  the  customer, 
DESC  has  climbed  the  logistical 
ladder  in  stock  availability  with- 
out an  across-the-board  invest- 
ment in  inventory. 

According  to  McBride,  DESC 
was  placed  under  stringent  fund- 
ing constraints  in  1973.  The 
challenge  of  keeping  shelves 
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filled  and  stock  available  at  a 
respectable  level  demanded  inno- 
vative management.  Following 
a series  of  deliberations,  DESC 
decided  to  “business-orient”  its 
inventot*y  management  process 
by  increasing  stock  levels  on 
high-demand  items  and  taking  a 
calculated  risk  with  less  material 
of  limited  interest. 

The  practice  was  instituted 
under  a frequency  management 
plan  known  as  the  DESC  Selec- 
tive Management  Category  Code 
Program.  Through  SMCC,  items 
with  the  greatest  influence  on 
DESC’s  business  were  identified, 
permitting  the  Center  to  meter 
its  investment  and  to  plan  and 
control  its  inventory  more  effi- 


ciently. The  ultimate  objective 
was  to  buy  most  of  what  sells 
best. 

Inventory  Categorization 

The  SMCC  Program  divided 
DESC’s  inventory  into  six  coded 
categories  of  items,  based  on  the 
number  of  requisitions  received 
and  the  annual  dollar  value  of 
sales  (see  Figure  1 on  p.  56). 
This  categorization  exercise  an- 
swered two  basic  questions  con- 
cerning the  activity  of  each 
DESC-managed  item : How 

often?  How  much? 

When  comparisons  were 

drawn,  the  items  in  categories 
A,  B,  C,  and  D,  the  high  and  me- 
dium frequency  items,  emerged 


as  the  items  providing  the  great- 
est portion  of  DESC’s  business 
activity  (see  Figure  2 on  p. 
57*).  Although  items  in  the  high 
and  medium  frequency  ranges 
represented  only  7 percent  of  the 
total  stocked  items  managed  by 
DESC,  they  accounted  for  a 
hefty  75  percent  of  the  requisi- 
tions and  74  percent  of  the  sales. 
Conversely,  the  SMCC  F items 
represented  about  92  percent  of 
the  stock,  yet  accounted  for  only 
25  percent  of  the  requisitions 
and  only  5 percent  of  the  sales. 

Recognizing  the  heavy  influ- 
ence of  SMCC  A and  B items  on 
DESC  business,  the  Center  be- 
gan profiling  these  items  to  es- 
tablish characteristic  patterns 
and  to  supply  the  management 
attention  needed  to  keep  SMCC 
A and  B shelves  filled.  Some  of 
the  more  pertinent  features  un- 
covered by  the  probe  about 
SMCC  A items,  both  individually 
and  collectively,  were  as  follows : 

• There  were  700  requisitions 
per  year  per  item. 

• 80  percent  of  the  items  were 
used  in  weapon  systems. 

• 20  percent  were  only  avail- 
able from  a single  source  of 
supply. 

• 70  percent  of  the  items  re- 
quired an  administrative  lead 
time  of  60  to  120  days. 

• 50  percent  of  SMCC  A’s  re- 
quired production  lead  times  of 
150  days  or  more. 

On  the  other  hand,  the  pro- 
filing process  disclosed  that 
SMCC  B’s  had  their  own  be- 
havior pattern : 

• As  a group,  the  items  had 
very  low  unit  prices. 


*The  vital  statistics  in  Figure  2 
were  used  by  DESC  as  a guideline  in 
establishing  its  SMCC  program.  They 
are  provided  here  so  that  parallels 
can  be  drawn  for  other  commodities 
as  well. 


A DESC  inventory  management  specialist  reviews  supply  control  data  relating 
to  one  of  the  174  SMCC-coded  connector  items  managed  on  a selective  basis. 
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Figure  1.  Coded  SMCC  Categories 
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All  organizational  levels  are  alerted  to  potential  problems 
through  DESC's  management  information  system. 


• There  were  360  requisitions 
per  year  per  item. 

• 80  percent  of  the  items  were 
used  in  weapon  systems. 

• Only  9 percent  were  iden- 
tified with  a sole  source  of  sup- 
ply. 

• 30  percent  reflected  an  ad- 
ministrative lead  time  of  60  to 
120  days. 

• 33  percent  had  production 
lead  times  of  150  days  or  more. 

In  essence,  the  profile  demon- 
strated that  SMCC  B’s  were  less 
expensive  and  created  less  fuss; 
even  though  the  requisition  vol- 
ume was  half  that  of  SMCC  A’s, 
B traffic  was  still  substantial. 
Therefore,  it  was  determined 
that  shepherding  B’s  as  well  as 
A’s  would  significantly  drive 
stock  availability  upward. 

Using  Money  and  Manpower 
Wisely 

The  crux  of  supply  manage- 
ment is  knowing  how  and  where 
to  invest  manpower  and  mone- 
tary resources.  Managerially, 
higher-grade  inventory  mana- 
gers are  assigned  A,  B,  C,  and 
E items,  while  mid-grade  mana- 
gers attend  to  D and  F items. 
This  permits  developed  skills  to 
be  used  to  the  fullest.  Financial 
resources  are  allocated  to  cate- 
gories according  to  essentiality 
factors  defined  by  the  DoD  vari- 
able safety-level  computations. 
Determined  through  operations 
research  simulation  studies, 
these  factors  have  the  effect  of 
weighting  the  amount  of  safety- 
level  investment  each  category 
receives. 

Because  of  their  more  predict- 
able demand  pattern,  SMCC  A 
and  B items  provide  a greater 
return  on  investment  in  terms 
of  stock  availability  than  the 
more  unpredictable  and  costly 
SMCC  E items.  Consequently, 


56 


October  1977 


DESC  puts  a greater  weight  on 
SMCC  A and  B safety-level  in- 
vestment through  the  use  of  the 
essentiality  factors  than  it  does 
on  the  SMCC  E items.  But  this 
isn’t  a case  of  “robbing  Peter  to 
pay  Paul’’  since  DESC  squeezes 
as  much  customer  support  as 
possible  from  every  supply  dol- 
lar spent.  This  allows  DESC  to 
maximize  its  total  system  stock 
availability  with  a minimum  in- 
vestment of  safety-level  dollars. 


Results  to  Date 

One  of  the  difficulties  in  man- 
aging an  inventory  of  DESC’s 
size  is  avoiding  an  overinvest- 
ment or  underinvestment  in 
stock.  While  the  situation  may 
appear  well-managed  across-the- 
board,  there  is  often  costly  over- 
investment on  one  item  and 
underinvestment  on  another. 
However,  through  SMCC,  DESC 
has  achieved  greater  parity  and 
is  much  closer  to  its  ultimate 
goal  of  paralleling  investment 
with  demand. 

In  terms  of  customer  support, 
SMCC  has  been  quite  effective 
in  giving  DESC  maximum  mile- 
age from  its  stock  fund  dollars. 
During  FY  75  and  FY  76,  stock 
availability  rose  from  88  per- 
cent to  94  percent,  while  back- 
orders fell  from  a record  high 
of  117,500  in  May  1974,  to  a 
record  low  of  39,200  in  July 
1976. 

Administrative  lead  times  on 
most  SMCC  categories  dropped 
markedly  the  last  year,  helping 
to  reduce  DESC’s  lead-time  in- 
vestment by  $7.4  million,  or  10.1 
percent.  The  trend  of  increasing 
administrative  lead  times  for 
SMCC  A items  was  also  re- 
versed, indicating  improved  cus- 
tomer support  on  these  items. 
Moreover,  with  SMCC’s  built-in 
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Ingredients  of  SMCC  Success 


Effective  combination  of  available  resources. 


sensitivity  to  changing  patterns, 
such  situations  never  occur  un- 
noticed. 

While  administrative  lead 
times  have  improved,  production 
lead  times  are  on  the  upswing 
for  some  of  DESC’s  stocked 
items,  a condition  generally 
attributed  to  the  recent  increase 
in  business  activity  throughout 
the  nation’s  economy.  SMCC  A, 
C,  and  E production  lead  times 
are  typically  longer  than  those 
in  the  SMCC  B,  D,  and  F cate- 
gories because  of  higher  unit 


costs  and  higher  dollar  value  of 
contracts  involved.  SMCC  E 
items  in  particular  have  en- 
countered stretched  lead  times, 
but  DESC  has  taken  no  actions 
to  stem  the  tide.  For  instance, 
SMCC  has  been  integrated  into 
the  appropriate  computer  file 
which  automatically  prints  the 
code  on  purchase  requests  in 
DESC’s  procurement  director- 
ate. This  permits  priority  proc- 
essing of  the  purchase  requests, 
thereby  minimizing  administra- 
tive lead  time  on  high  and  med- 


ium frequency  items.  Likewise, 
item  managers  have  been  re- 
quested to  expedite  delivery  on 
contracts  for  SMCC  A,  B,  C,  and 
D items  which  may  prove  trou- 
blesome. Finally,  SMCC  B and  D 
items  have  been  assigned  to  spe- 
cial item  managers  who  employ 
selective  management  techniques 
which  permit  them  to  exercise 
more  adequate  control  over  lead 
times. 

Intensity,  Sensitivity 

While  DESC’s  Selective  Man- 
agement Code  Category  Pro- 
gram follows  the  basic  formula 
of  other  inventory  management 
concepts,  SMCC  is  earmarked 
by  its  intensity  and  sensitivity. 
Through  the  comprehensive 
structure  of  the  program,  DESC 
today  knows  a great  deal  more 
about  the  variables  of  its  sup- 
ply business  than  it  did  just  two 
years  ago.  And  it  is  applying  the 
ingredients  for  SMCC  success — 
people,  money,  and  inventory — 
accordingly. 

Whether  to  improve  support 
or  to  circumvent  a knotty  prob- 
lem, numerous  strategies  are 
available  to  item  managers.  But 
it  is  essential  that  they  first 
know  what  knobs  to  turn. 
Through  SMCC,  the  Defense 
Electronics  Supply  Center  has 
introduced  just  such  a manage- 
ment discipline. 
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A Retrospective  Look 


The  Yesterday,  Today,  and  Tomorrow 
of  integrated  Logistics  Support 


In  its  thirteenth  year,  ILS  is  surely  here  to  stay;  the  task  for 
the  ILS  community  is  to  make  that  stay  a productive  one. 


tt  hile  most  of  the  aerospace  industry  knows 
about  integrated  logistics  support,  it  is  a much 
smaller  number  of  practitioners  that  realize 
ILS  is  well  into  its  second  decade.  Although  ILS 
has  changed  in  form  since  its  inception  in  1964, 
the  objective  remains  to  strike  an  optimum 
balance  between  total  system  performance,  cost, 
and  schedule  while  developing  an  integrated 
support  system.  This  article  looks  at  the  evo- 
lution of  the  ILS  concept,  the  progress  made 
in  implementing  ILS,  and  the  challenges  still 
to  be  met. 

The  Birth  of  ILS 

On  June  19,  1964,  DoD  Directive  4100.35, 
“Development  of  Integrated  Logistics  Support 
for  Systems  and  Equipment,”  was  issued  to 
assure  that  “effective  logistics  support  for  sys- 
tems and  equipments  is  systematically  planned, 
acquired,  and  managed  as  an  integrated  whole 
. . . to  obtain  maximum  materiel  readiness  and 
optimum  cost  effectiveness.”  The  directive  was 
developed  by  the  DoD  Equipment  Maintenance 
and  Readiness  Council,  assisted  by  the  Main- 

by  Maj.  Ned  H.  Criscimagna,  USAF 
Directorate  of  Maintenance,  Engineering  & Supply 
ODCS  for  Systems  and  Logistics 
Department  of  the  Air  Force 


tenance  Advisory  Committee  of  the  National 
Security  Industrial  Association.  This  joint 
DoD-industry  effort  made  the  directive  rather 
unique  and  demonstrated  the  universal  concern 
for  effective  and  economical  support  systems. 
Although  this  concern  had  long  been  voiced 
by  logisticians,  DoDD  4100.35  represented  the 
first  official  move  to  improve  the  development 
of  logistic  support  systems  and  as  such  was  a 
milestone  in  defense  system  programming. 

As  with  all  new  ideas,  ILS  was  greeted  with 
mixed  emotions.  DoD  and  the  defense  industry 
had  been  saturated  with  “integrated  systems,” 
“systems  engineering,”  “cost  effectiveness,” 
“system  effectiveness,”  and  a seemingly  end- 
less list  of  new  disciplines  and  management 
philosophies.  There  were  many  who  felt,  or 
hoped,  that  ILS  would  be  just  another  buzz 
word  that  would  soon  be  discarded.  Others  saw 
the  need  for  such  an  approach  to  support  plan- 
ning, but  dreaded  the  inevitable  deluge  of  new 
regulations,  reports,  and  associated  paperwork. 
ILS,  however,  did  not  go  out  of  fashion  or  result 
in  an  unmanageable  mountain  of  ink  and  paper. 
It  wasn’t  long  before  the  song  of  ILS  was  being 
sung  from  every  professional  circle  within  the 
defense  community. 

One  of  the  first  major  forums  for  discussing 
integrated  logistics  support  was  the  first  Elec- 
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tronics  Industries  Association  Conference  on 
Systems  Effectiveness  in  October  1965.  C.  W. 
Winkler  of  Douglas  Aircraft  presented  a paper 
which  highlighted  the  seven  basic  elements  of 
the  ILS  concept  (Figure  1)  as  defined  in  DoDD 
4100.35.  Some  of  the  key  points  were: 

• ILS  is  necessary  for  the  development  of 
an  effective  and  economical  support  system. 

• For  the  most  part,  the  cost  of  ownership 
of  weapon  systems  far  exceeds  the  development 
and  investment  costs. 

• The  cost  of  ownership  of  weapon  systems 
is  most  effectively  controlled  by  emphasis  on 
ILS  as  early  in  the  conceptual  phase  of  the 
system  as  possible. 

• ILS  represents  the  start-to-finish  life- 
cycle  planning  of  total  maintenance  and  logis- 
tics support  of  weapon  systems. 

The  next  several  years  saw  a continuing  in- 
terest in  and  further  refinements  of  the  ILS 
concept.  At  the  First  Annual  Logistics  Manage- 
ment Symposium,  George  J.  Vecchietti  of  the 
National  Aeronautics  and  Space  Administra- 
tion gave  a first-hand  view  of  the  contract 
strategy  then  being  used  or  being  considered 
for  use  by  NASA  in  the  logistics  support  area. 
Award-fee  contracts  and  early,  meaningful 
pricing  of  logistics  support  line  items  were  the 
two  objectives  NASA  was  pursuing  to  improve 
logistics  support  management.  At  the  Second 
Systems  Effectiveness  Conference,  Ben  S. 
Blanchard  of  General  Dynamics  told  of  the 
interrelationship  of  cost  effectiveness,  system 
effectiveness,  and  integrated  logistics  support. 

^^The  cost  of  ownership  of  weapon 
systems  Is  most  effectively  con- 
trolled by  emphasis  on  ILS  as  early 
In  the  conceptual  phase  of  the  sys- 
tem as  possible/' 

And  at  the  Sixth  Annual  Reliability  and  Main- 
tainability Conference,  John  E.  Losee  addressed 
the  development  of  quantitative  logistics  per- 
formance parameters  during  the  conceptual  and 
contract  definition  phase  of  Air  Force  pro- 
grams. 

These  men  and  countless  others  helped  to 
better  define  ILS  and  to  develop  procedures  and 
methodology  for  implementing  the  concept.  In 
those  formulative  years,  the  defense  commu- 
nity tried  new  and  innovative  management 


techniques  in  order  to  put  ILS  into  the  devel- 
opment process. 

The  first  large-scale  implementation  of  ILS 
involved  the  B-1  and  F-15  programs.  The  B-1 
ILS  office  was  established  within  the  program 
office  as  a directorate  coequal  with  the  other 
“standard”  directorates  (such  as  engineering, 
test,  and  program  control).  The  majority  of 
the  officers  hand-picked  for  the  ILS  team  had 
engineering  degrees  and  experience  as  main- 
tenance officers  or  logistics  officers.  Responsi- 
bilities were  assigned  according  to  a three- 
dimensional  matrix  (Figure  2 shows  a repre- 
sentative segment),  with  each  officer  given 
several  responsibilities  in  three  areas : logistics 
elements,  subsystem  design,  and  logistics  tasks. 

"Although  ILS  has  changed  In  form 
since  Its  Inception  In  1964,  the  ob- 
jective remains  to  strike  an  opti- 
mum balance  between  total  system 
performance,  cost,  and  schedule 
while  developing  an  Integrated 
support  system." 

First  regarded  as  just  an  Air  Force  Logistics 
Command  liaison  office,  the  ILSO  was  soon  ac- 
cepted as  a working  member  of  the  B-1  team 
and  was  given  program  responsibility  and 
authority  for  the  acquisition  of  the  support 
system.  None  of  this  could  have  happened,  how- 
ever, if  the  program  manager  had  not  recog- 
nized the  importance  of  proper  logistics  plan- 
ning and  afforded  ILS  the  same  attention  as 
was  given  to  the  other  program  functions.  This 
same  visibility  was  also  afforded  ILS  in  the 
F-15  program,  where  the  ILSO  had  the  full 
support  of  its  program  manager  and  was  also 
manned  by  a select  group  of  individuals  with 
engineering  and  support  backgrounds. 

AFALD  Estobllshed 

Such  efforts  to  put  ILS  into  practice  were 
repeated  in  many  other  programs,  both  large 
and  small.  But  the  ILS  effort  was  not  restricted 
to  program-level  management.  AFLC  was  con- 
tinually assessing  its  role  in  system  acquisition 
and  saw  a need  for  more  active  involvement. 
In  April  1974,  AFLC  established  a Deputy 
Chief  of  Staff  for  Acquisition  Logistics  re- 
sponsible for  focusing  command  management 
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attention  and  resources  on  procedures,  tech- 
niques, and  activities  necessary  to  implement 
fully  the  ILS  concept.  Two  years  later,  the 
Chief  of  Staff  directed  the  establishment  of  the 
Air  Force  Acquisition  Logistics  Division.  Oper- 
ating as  an  AFLC  subcommand,  AFALD  has 
the  responsibility  and  authority  to  provide 
strong,  constructive  advocacy  for  controlling 
life  cycle  costs,  and  to  assist  Air  Force  Systems 
Command  program  managers  during  all  phases 
of  the  system  acquisition  process. 

'The  concept  of  making  early,  rela- 
tively small  investments  in  order 
to  realize  a lower  life  cycle  cost  is 
central  to  the  ILS  philosophy." 

Compared  to  the  early  days  of  ILS,  AFALD 
represents  a considerable  change  in  the  man- 
agement attention  and  manpower  resources 
dedicated  to  logistics  support.  Yet  organiza- 
tional changes  have  not  been  the  only  outgrowth 
of  ILS;  DoD  Directive  4100.35,  supplemented 
by  DoD  Guide  4100. 35G,  has  been  updated  and 
has  spawned  several  Air  Force  logistics  docu- 
ments, including  an  ILS  implementation  guide 
and  a regulation  dealing  with  acquisition  of 
support  equipment.  In  addition,  the  develop- 
ment of  policy  and  procedures  for  related  sub- 
jects, such  as  life  cycle  costs,  has  improved  the 
implementation  of  ILS  in  systems  programs. 
Today,  the  original  7 elements  of  ILS  have  been 
expanded  to  11,  all  major  programs  have  an 
ILS  office,  and  both  AFSC  and  AFLC  continue 
to  improve  the  management  and  application  of 
ILS. 

Despite  the  progress  made  since  1964,  there 
still  remains  much  to  be  done.  Some  of  the 
early  misgivings  about  ILS  persist,  and  some 
program  managers  still  have  to  be  “sold”  on 
ILS.  This  selling  job  is  necessary  not  to  over- 
come resistance  to  good  support  planning,  but 
because  of  the  current  management  and  budget- 
ing environment.  Take,  for  example,  the  budg- 
etary pressures  put  on  the  program  manager. 
Asked  to  plan  several  years  into  the  future, 
he  is  given  money  one  year  at  a time;  but 
the  exact  amount  is  not  known  until  the  budget 
is  approved  by  Congress.  Months  of  careful 
planning  at  the  program  level  can  be  negated 
when  the  upcoming  fiscal-year  budget  is  cut. 
This  in  turn  may  require  reprogramming,  de- 


Figure  1.  Basic  Elements  of  ILS 

• Planned  maintenance 

• Spares  & repair  parts 

• Support  equipment 

• Technical  logistics  data  & information 

• Contract  maintenance 

• Logistics  support  personnel 

• Facilities 

laying,  reducing,  or  eliminating  certain  tasks. 
Deciding  which  tasks  are  to  be  affected  is  nor- 
mally done  on  the  basis  of  immediate  need; 
that  is,  those  tasks  needed  today,  such  as  hard- 
ware design,  are  kept  constant  at  the  expense 
of  tasks  which  do  not  have  a first-hand  effect 
on  the  program.  Such  tasks  include  ILS,  reli- 
ability demonstration,  development  of  technical 
manuals,  and  related  “supporting”  tasks.  Iron- 
ically, these  areas  with  no  immediate  effect  on 
the  program  are  the  determining  factors  in 
the  downstream  operating  and  support  costs. 

All  too  often  one  is  forced  to  live  within 
today’s  budget  at  the  expense  of  higher  life 
cycle  costs.  The  concept  of  making  early,  rela- 
tively small  investments  in  order  to  realize  a 
lower  life  cycle  cost  is  central  to  the  ILS  phil- 
osophy. Only  when  ILS  is  implemented  early 
and  afforded  a chance  to  impact  design  can  the 
intent  of  ILS  be  met.  Without  this  front-end 
investment,  ILS  can  often  do  no  more  than 
develop  the  least-expensive  support  system  for 
a patently  unsupportable  weapon  system. 

Another  obstacle  to  selling  ILS  is  the  desire 
of  every  manager  to  “be  certain”  before  making 

Figure  2.  Responsibility  Matrix 
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a decision.  All  managers  would  prefer  to  have 
all  the  facts,  eliminating  all  uncertainty  be- 
fore making  a decision.  A reality  of  life,  how- 
ever, is  that  uncertainty  is  always  present,  and 
we  never  have  the  time  or  the  ability  to  know 
all  the  facts.  We  therefore  must  rely  on  ana- 
lytical techniques  and  our  own  experience  to 
minimize  the  effect  of  uncertainty,  and  thereby 
reduce  the  risk  of  a bad  decision. 


^'Another  obstacle  to  selling  ILS  is 
the  desire  of  every  manager  to  'be 

certain'  before  making  a decision 

The  analytical  tools  developed  in 
the  area  of  ILS,  such  as  logistics  sup- 
port analysis,  optimum  repair-level 
analysis,  and  various  cost  models, 
are  reasonably  accepted  as  mean- 
ingful and  useful." 


The  analytical  tools  developed  in  the  area  of 
ILS,  such  as  logistics  support  analysis,  optimum 
repair-level  analysis,  and  various  cost  models, 
are  reasonably  accepted  as  meaningful  and  use- 
ful. Still,  the  validity  of  any  analysis  is  largely 
dependent  on  the  accuracy  of  the  data  used. 
Herein  lies  the  dilemma  of  the  decision  maker 
regarding  ILS : the  early  phases  of  a program, 
when  the  ILS  analysis  is  most  effective  in  influ- 


encing design,  are  the  times  when  the  data  is 
very  “soft.”  The  program  manager  is  reluctant 
to  implement  a design  change  at  a cost  of,  say, 
$100,000,  simply  because  ILS  analysis  based  on 
soft  data  indicates  an  LCC  savings  of  $1  million 
over  the  present  design.  If  the  decision  is  de- 
layed until  the  data  is  “dependable,”  change 
may  be  impractical  because  of  the  implementa- 
tion cost  and  schedule  impact.  The  net  result 
is  that  many  of  these  LCC-affecting  changes  are 
not  made. 

The  problems  of  budget  constraints  and  good 
data,  however,  are  secondary  compared  to  the 
lack  of  adequately  manned  ILS  offices.  This 
manning  inadequacy  consists  of  two  types: 
quantity  and  experience.  Many  ILS  offices  do 
not  have  the  recommended  allotment  of  person- 
nel to  develop  and  acquire  a supportable  weapon 
system  together  with  a complete,  cost-effective 
support  system.  Except  for  very  small  pro- 
grams, a smaller  number  of  people  cannot  ade- 
quately define,  develop,  and  manage  an  effective 
ILS  program. 

The  number  of  ILS  personnel  could  be  kept 
at  a minimum  if  all  were  experienced  both  in 
program  management  and  ILS,  but  there  is  a 
discouragingly  small  number  of  these  people 
and  many  of  them  move  into  other  career  areas 
after  a few  years  in  an  ILSO.  The  average  pro- 
gram therefore  begins  with  only  a few,  inex- 
perienced people  to  plan  and  manage  the  ILS 
program.  With  the  current  emphasis  on  austere 
manning  and  the  need  for  career  diversification 
(to  be  competitive  for  promotion),  the  people 
problem  is  not  a temporary  situation. 

Solutions  Are  Not  Simple 

Solutions  to  these  obstacles  are  more  easily 
stated  than  solved,  but  there  are  some  basic 
steps  available  to  improve  the  immediate  situ- 
ation and  eventually  eliminate  these  obstacles. 
First,  the  ILS  community  must  I’ecognize  and 
appreciate  the  real-world  constraints  placed  on 
the  program  manager,  who  is  tasked  to  bring 
in  a program  on  schedule  and  within  a budget 
which  he  only  knows  a year  at  a time.  With 
minimal  discretionary  funds  available  to  him, 
he  is  asked  to  explore  alternatives  and  change 
the  program  plan  as  needed  in  order  to  mini- 
mize life  cycle  costs  and  achieve  the  operational 
requirements.  He  is  asked  to  request  more  front- 
end  money  (he  is  judged,  in  part,  by  his  ability 
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There  must  be  a genuine 
effort  on  the  part  of  all 
concerned  to  make  I LS 
a success 


Opportunities  for  promotion, 
education,  and  job  advance 
ment  in  the  ILS  career  fiel 
must  be  fostered  by  top 
management 


Training  of  ILS  personnel 
must  not  only  continue,  but 
be  improved  as  well 


The  ILS  community  must 
recognize  the  real-world 
constraints  put  on  the 
program  manager 


Solutions  to 
ILS  Obstacles; 


to  avoid  cost  growth)  to  save  dollars  in  the 
long  run,  even  though  he  will  undoubtedly  be 
somewhere  else  by  then.  One  need  not  be  imagi- 
native to  guess  which  direction  the  program 
manager  is  inclined  to  go — control  today’s  costs ! 

A second  step  is  to  continue  and  improve  the 
training  of  ILS  personnel.  The  Air  Force’s 
Logistics  and  Systems  Commands  currently 
have  a program  which  allows  officers  in  one 
command  to  gain  experience  in  the  other  and 
then  return  to  their  parent  command.  As  a re- 
sult of  this  cross-fertilization,  the  future  pro- 
gram manager  gains  logistics  experience  and 
the  future  deputy  program  manager  gains  ex- 
perience in  the  total  program  management  area. 
Eventually,  some  of  the  harmful  command  pa- 
rochialism should  disappear  and  ILS  should 
become  more  of  a team  effort. 


all  concerned  to  make  ILS  work!  This  includes 
the  program  manager,  the  air  staff,  the  devel- 
oping and  supporting  commands,  and  the  Con- 
gress. The  program  manager  must  have  the 
courage  to  make  the  hard  decisions,  and  higher 
management  must  encourage  and  support  the 
program  manager  in  such  decisions.  The  com- 
mands must  make  ILS  a true  objective,  not  at 
the  expense  of  other  requirements,  but  as  an 
equally  important  and  vital  requirement.  Con- 
gress must  understand  and  encourage  the  ILS 
effort  and  should  consider  alternate  funding 
approaches  for  system  development  and  acqui- 
sition (such  as  multiyear  funding). 

Integrated  Logistics  Support  has  come  a long 
way  over  the  past  13  years.  It  is  a concept  that 
is  here  to  stay.  The  task  for  all  involved  is  to 
make  that  stay  a productive  one. 


'MLS  is  necessary  for  the  develop- 
ment of  an  effective  and  economical 
support  system." 

A third  step  is  to  make  the  ILS  career  field 
an  attractive  one.  If  ILS  is  to  compete  with 
more  glamourous  career  fields  for  quality,  then 
opportunities  for  promotion,  education,  and  job 
advancement  must  be  fostered  by  the  highest 
management  levels.  There  has  been  much  prog- 
ress in  this  area,  and  the  old  fear  of  “being 
stuck  in  logistics’’  no  longer  is  prevalent. 

Finally,  the  obstacles  to  ILS  can  be  elimi- 
nated through  a genuine  effort  on  the  part  of 


MAJOR  NED  H.  CRISCIMAGNA,  USAF,  is  a Re- 
liability and  Maintainability  Staff  Officer  assigned  to 
the  Logistics  Engineering  and  Support  Division,  Di- 
rectorate of  Maintenance,  Engineering  & Supply, 
under  the  Deputy  Chief  of  Staff  for  Systems  and 
Logistics. 

He  was  previously  assigned  to  the  Directorate  of 
Aerospace  Logistics,  part  of  the  Air  Force  Acquisition 
Logistics  Division. 

Major  Criscimagna  has  a B.S.  degree  from  the  Uni- 
versity of  Nebraska  and  a Master’s  degree  from  the 
Air  Force  Institute  of  Technology.  He  is  also  a gradu- 
ate of  the  Air  Force  Command  and  Staff  College,  and 
has  completed  the  National  Security  Management 
Course  conducted  by  the  Industrial  College  of  the 
Armed  Forces. 


Defense  Management  Journal 


63 


Who’s  New  in  DoD 

Their  decisions  will  have  significant  impact  on  Department  of  Defense  planning  and 
policies.  For  the  most  part,  however,  their  names  and  faces  are  relatively  little  known  to 
persons  in  DoD.  In  this  issue,  the  Journal  offers  brief  introductions  to  newly  appointed 
managers  in  the  Military  Departments. 


Robert  L.  Nelson  is  the  Assist- 
ant Secretary  of  the  Army  (Man- 
power and  Reserve  Affairs).  When 
nominated,  he  was  president  of 
the  Broadcasting  Division  of 
Washington  Star  Communica- 
tions, Incorporated,  which  he  had 
joined  in  1974  as  the  newspaper’s 
senior  vice  president  (treasurer 
and  counsel). 

He  was  employed  by  various 
government  organizations  from 
1958  to  1966,  serving  consecu- 
tively with  the  U.S.  Commission 
on  Civil  Rights,  the  U.S.  Agency 
for  International  Development, 
and  the  U.S.  AID  Mission  to 
Brazil.  Subsequently,  before  join- 
ing Washington  Star  Communica- 
tions, he  served  in  legal  and  man- 
agement capacities  in  private 
industry. 

Mr.  Nelson,  46,  received  a B.A. 
degree  from  Bates  College, 
Maine,  and  a law  degree  from 
Georgetown  University  Law  Cen- 
ter. He  is  a member  of  the  Bar 
of  the  District  of  Columbia. 


David  E.  Mann  is  the  Assistant 
Secretary  of  the  Navy  (Research 
and  Development). 

From  1951  to  1965,  Dr.  Mann 
held  positions  with  the  National 
Bureau  of  Standards,  first  as 
physicist  and  later  as  chief  of 
molecular  spectroscopy.  In  1957, 
he  was  awarded  both  Guggen- 
heim and  Fulbright  fellowships 
for  study  abroad. 

He  joined  the  Advanced  Re- 
search Products  Agency  in  1966, 
and  in  1967  he  was  appointed 
deputy  director  of  that  agency’s 
Ballistic  Missile  Defense  Office. 

In  1968  he  became  director  of 
the  Strategic  Technology  Office, 
where  he  initiated  development 
programs  in  high-energy  lasers, 
long-range  infrared  and  radar 
sensors,  and  undersea  and  ASW 
sensor  technology. 

Dr.  Mann  became  special  as- 
sistant for  Navy  advanced  system 
projects  to  the  Chief  of  Naval 
Operations  in  1973. 

He  holds  a Ph.D.  in  chemistry 
from  the  University  of  Chicago. 


Alan  J.  Gibbs,  the  Assistant 
Secretary  of  the  Army  (Installa- 
tions, Logistics  and  Financial 
Management),  had  been  deputy 
commissioner  of  the  New  Jersey 
Department  of  Human  Services 
since  1974. 

He  began  his  career  in  public 
service  in  1963,  when  he  entered 
the  management  intern  program 
with  the  National  Labor  Relations 
Board  as  a labor-management  re- 
lations examiner  in  Baltimore. 

In  1966,  Mr.  Gibbs  joined  the 
Equal  Employment  Opportunity 
Commission  in  Washington,  DC. 
In  1968,  he  was  appointed  area 
director  for  Alabama  and  Ten- 
nessee. 

Mr.  Gibbs,  39,  was  assist- 
ant health  services  administrator 
in  the  New  York  City  Health  Serv- 
ices Administration  from  1970  to 
1972,  when  he  became  first  dep- 
uty commissioner  of  health. 

He  holds  a B.S.  degree  in  man- 
agement and  an  M.S.  degree  in 
labor  and  industrial  relations 
from  the  University  of  Illinois. 


Antonia  Handler  Chayes  is  the 

Assistant  Secretary  of  the  Air 
Force  (Manpower,  Reserve  Affairs 
and  Installations).  At  the  time  of 
her  nomination,  she  was  a part- 
ner in  the  Boston  law  firm  of 
Csaplar  and  Bok. 

Her  professional  experience  in- 
cludes the  deanship  of  Jackson 
College,  Tufts  University.  She  was 
a lecturer  in  law  at  Yale  and  Bos- 
ton University  law  schools. 

Ms.  Chayes  served  as  Director 
of  Education  and  Urban  Develop- 
ment for  Action  for  Boston  Com- 
munity Development,  and  as  liai- 
son for  the  Boston  Model  Cities 
Administration.  She  was  also  a 
member  of  the  planning  staff  of 
the  National  Institute  of  Mental 
Health. 

She  was  a member  of  the  White 
House  staff  in  1961  and  1962. 

Ms.  Chayes,  48,  received  an 
A.B.  degree  in  government  from 
Radcliffe  College  and  a J.D.  de- 
gree from  George  Washington 
University. 


Edward  Hidalgo  is  the  Assist- 
ant Secretary  of  the  Navy  (Man- 
power, Reserve  Affairs  and  Lo- 
gistics). 

Mr.  Hidalgo  was  a legal  advisor 
with  the  State  Department  from 
1942  to  1943. 

He  was  a member  of  the  United 
States  Delegation  to  the  Inter- 
American  Conference  in  Bogota, 
Columbia,  in  1948. 

From  1965  to  1966,  he  was 
special  assistant  to  the  Secre- 
tary of  the  Navy,  and  from  1966 
to  1972,  he  directed  the  Euro- 
pean office  of  the  law  firm  of 
Cahill,  Gordon,  and  Reindel. 

In  1972,  Mr.  Hidalgo  served  as 
special  assistant  for  economic 
affairs  to  the  director  of  the 
United  States  Information  Agen- 
cy, and  in  1973  he  became  gen- 
eral counsel  of  the  agency. 

He  holds  a B.A.  degree  from 
Holy  Cross  College,  a J.D.  degree 
from  Columbia  Law  School,  and 
a degree  in  civil  law  from  the 
University  of  Mexico. 


Percy  A.  Pierre,  the  Assistant 
Secretary  of  the  Army  (Research 
and  Development),  was  dean  of 
the  School  of  Engineering,  How- 
ard University,  at  the  time  of  his 
appointment.  From  1968  to  1971, 
he  was  a research  engineer  with 
the  Rand  Corporation. 

Dr.  Pierre,  38,  was  a White 
House  Fellow  from  1969  to  1970. 
From  1973  to  1975,  he  divided 
his  time  between  Howard  Uni- 
versity and  the  Alfred  P.  Sloane 
Foundation. 

He  has  been  a member  of  the 
Board  of  Directors  of  the  Engi- 
neering Colleges  Council;  Ameri- 
can Society  for  Engineering  Edu- 
cation; and  the  Advisory  Panel  for 
the  Institute  of  Applied  Technol- 
ogy, National  Bureau  of  Stand- 
ards. He  served  as  consultant  to 
the  Center  for  Naval  Analyses. 

He  holds  B.S.  and  M.S.  de- 
grees in  electrical  engineering 
from  the  University  of  Notre 
Dame  and  a Ph.D.  degree  in  elec- 
tric engineering  from  Johns  Hop- 
kins University. 


a few  words  about.. 


Technological  Breakthrough 


DLA  Develops  Laser  Labeling  System 


The  Defense  Logistics  Agen- 
cy’s Defense  Depot  Ogden,  Utah, 
has  developed  a system  which 
uses  a high-powered,  pulsed 
laser  beam  controlled  by  a mi- 
crocomputer to  burn  labels  di- 
rectly on  shipping  containers. 
Known  as  the  container  automat- 
ic marking  system  (CAMS),  the 
new  method  created  by  DLA  en- 
gineers has  dramatically  in- 
creased the  speed,  efficiency,  and 


accuracy  of  marking  packaged 
material  for  shipping. 

The  CAMS  unit  is  interfaced 
through  a microprocess  compu- 
ter to  the  depot’s  computer  sys- 
tem. The  computer  automatically 
gathers  the  weight  and  cube  data 
and  assigns  the  package  to  a 
shipment  sorter  line.  When  the 
shipment  unit  on  a particular 
line  is  complete,  it  is  released  to 
CAMS  for  automatic  marking. 


Fundamentals  Offered 

New  Course  Teaches  OR/SA  Skills 


A 12-week  course  on  the  fun- 
damentals of  operations  re- 
search/systems analysis  is  being 
offered  by  the  Army  Logistics 
Management  Center,  Fort  Lee, 
Virginia.  The  course  is  for  Army 
officers  who  have  the  OR/SA 
specialty  but  do  not  have  a grad- 
uate degree  in  either  OR/SA  en- 
gineering or  OR/SA  business. 

The  course  is  designed  to  give 
Army  officers  an  understanding 
of  OR/SA  theory  and  method- 
ologies. It  will  provide  the  stu- 
dent with  the  knowledge  and 
skills  required  in  the  military 
environment  to  perform  and 
critically  evaluate  OR/SA  stud- 


ies, interpret  them  for  decision 
makers,  and  communicate  effec- 
tively with  systems  analysts  and 
other  specialists. 

The  next  class  of  the  OR/SA 
Military  Applications  Course 
will  begin  February  6. 

ALMC  also  is  offering  a series 
of  short,  intensive  seminars  to 
provide  experienced  OR/SA  offi- 
cers with  information  they  may 
not  have  received  during  their 
formal  education  process. 

For  further  information,  write 
the  Chairman,  Systems  and  Cost 
Analysis  Department,  School  of 
Logistics  Science,  ALMC,  Fort 
Lee,  VA  23801.  Phone:  (804) 
734-1231. 


Upon  signal,  CAMS,  by  con- 
trolled laser  burning,  prints  the 
destination  address  directly  on 
the  shipping  container  surface. 

The  new  procedure  eliminates 
several  human  steps  and  accom- 
plishes the  work  20  times  faster 
than  old  hand-marking  methods. 

Estimated  costs  for  a new 
CAMS  system  are  about  $95,000. 
Anticipated  annual  savings  will 
be  $39,761  on  the  basis  of  650, 
000  packages  marked. 


DMJ  Rated 
Outstanding 

The  Defense  Management 
Journal  recently  received  a 
Blue  Pencil  Award  for  edi- 
torial excellence  from  the 
National  Association  of 
Government  Communica- 
tors. The  award  was  be- 
stowed in  competition  with 
other  government  publica- 
tions whose  professional 
staffs  are  comparable  in  size 
to  that  of  the  Journal. 

The  NAGC  is  the  pro- 
fessional organization  for 
information  and  editorial 
people  at  all  levels  of 
government — federal,  state, 
and  local. 
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ASPR  Discussed 


Job  Guide  Study 

New  Handbooks  Simplify  Maintenance 


Proposed  Bill  Modifies 
Procurement  Regulations 

A recently  introduced  Senate 
bill,  “Federal  Acquisition  Act  of 
1977”  (S  1264),  would  have  far- 
reaching  effects  on  defense  pro- 
curement. The  bill  contains  rec- 
ommendations for  eliminating 
the  separate  DoD  procurement 
regulations  (ASPR),  requiring 
functional  specifications  and  life 
cycle  costing  for  procurements 
in  addition  to  major  systems, 
giving  final  authority  to  the 
General  Accounting  Office  in  bid 
protests,  restricting  the  use  of 
competition  and  sealed  bids,  ex- 
panding the  requirements  for 
cost  data  for  sole-source  pro- 
curements, and  introducing  a 
requirement  for  price  data  for 
all  large  purchases. 


Executive  Reorganization 

Personnel  Programs 
Under  Review 

President  Carter  has  estab- 
lished the  “Federal  Personnel 
Management  Project”  as  part  of 
his  plan  to  reorganize  the  Execu- 
tive Branch.  Under  the  direction 
of  Alan  Campbell,  chairman  of 
the  Civil  Service  Commission, 
and  Wayne  Grandquist  of  the 
Office  of  Management  and 
Budget,  the  project  will  estab- 
lish eight  special  task  forces  to 
study  and  recommend  changes  in 
the  government’s  civil  service 
and  personnel  systems.  The  en- 
tire spectrum  of  personnel  pro- 
grams is  planned  for  review. 


The  Advanced  Systems  Divi- 
sion of  the  Air  Force  Human 
Resources  Laboratory  recently 
completed  study  of  a new  series 
of  aircraft  maintenance  instruc- 
tion guides  designed  to  simpli- 
fy maintenance  troubleshooting 
while  reducing  training  time  for 
aircraft  maintenance  specialists. 

It  has  generally  been  accepted 
that  technical  orders  are  hard 
to  use  during  aircraft  mainte- 
nance operations  and  that  tech- 
nicians often  have  difficulty  find- 
ing the  specific  information  they 
require. 

The  new  guides  are  small, 
four-  by  eight-inch  handbooks  of 
sequential  instructions  for  main- 
tenance and  troubleshooting, 
convenient  for  use  on  the  flight- 
line or  in  the  maintenance  area. 
Emphasis  is  placed  on  providing 
all  the  information  that  the  tech- 
nician requires  to  do  the  job  in 


$25,000  Incentive  Award 


President  Carter  recently  pre- 
sented a $25,000  incentive  award 
to  Lawrence  L.  Guzick,  head  en- 
gineer in  the  Aircraft  Carrier 
Ship  Logistics  division  of  the 
Naval  Sea  Systems  Command. 

Mr.  Guzick  won  the  award  for 
his  invention  of  an  orifice  and 
filter  assembly  which  replaces 
the  conventional  steam  trap  used 
on  naval  ships  since  early  in  the 
development  of  steam-powered 
equipment.  The  device,  which 
has  been  approved  for  fleet-wide 
installation,  improves  the  relia- 
bility and  availability  of  steam- 


one  place  and  ensuring  that  it 
is  written  in  clear  and  easily 
understandable  terms. 

Advantages  of  the  job  guides 
include  their  small  size,  their 
readability,  and  their  under- 
standability.  The  study  showed 
that  the  maintenance  technicians 
found  the  illustrations  keyed  to 
specific  procedures  useful.  The 
inclusion  of  job  preparation  in- 
formation was  seen  as  a method 
of  reducing  the  time  required 
for  some  tasks  and  of  contrib- 
uting to  maintenance  safety. 

One  disadvantage  found  dur- 
ing the  study  was  that  for  some 
maintenance  tasks,  technicians 
had  to  use  too  many  volumes. 
Sometimes,  too,  they  had  diffi- 
culty locating  specific  informa- 
tion quickly. 

The  study  findings  will  be  used 
in  the  preparation  of  future 
guides. 


machinery  systems,  reduces 
steam  leakage,  and  improves  tur- 
bine and  thermal  efficiency  of 
the  entire  power  system  of 
steam-powered  ships.  The  Navy 
anticipates  annual  fuel  and 
maintenance  cost  savings  of  over 
$10  million  from  Mr.  Guzick’s 
invention. 

The  $25,000  award — the  maxi- 
mum that  can  be  granted — was 
conferred  on  only  five  previous 
occasions  since  the  Federal  In- 
centive Awards  program  was 
established  in  1954,  and  never 
before  by  a President. 


President  Honors  Federal  Employee 
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Feedback 


To  the  Editor: 

The  article  “Tracking  the 
Costs  of  Depot  Mainte- 
nance” (July  1977)  stated 
that  on  October  1,  1976,  the 
Army  and  Navy  imple- 
mented the  depot  mainte- 
nance cost  accounting  sys- 
tem prescribed  by  DoD 
7220. 29H  and  that  the  Air 
Force  plans  full  implemen- 
tation by  October  1979. 
Taken  at  face  value,  these 
statements  could  lead  one  to 
conclude  that  the  Army  and 
Navy  have  fully  and  suc- 
cessfully implemented  the 
DoD  cost  accounting  system 
while  the  Air  Force  has  not 
implemented  any  portion  of 
it.  Such  conclusions  would 
not  be  valid. 

The  following  informa- 
tion is  provided  to  supple- 
ment the  text  of  the  article 
and  to  correct  any  erron- 
eous conclusions  which 
might  have  been  drawn 
since  it  was  published. 

The  Defense  Audit  Serv- 
ice reports  the  Army  and 
Navy  have  not  fully  imple- 
mented the  DoD  depot 
maintenance  cost  account- 
ing system.  In  a July  26, 
1977  draft  report,  the  DAS 
stated  the  “system  was  not 
fully  implemented  within 
the  11  Army  maintenance 
facilities.”  Further,  the 


“system  was  not  fully  and 
uniformily  implemented  at 
Naval  Shipyards.”  Even  at 
the  Naval  Air  Rework  Fa- 
cilities, in  which  the  “Hand- 
book was  generally  imple- 
mented,” the  DAS  identi- 
fied several  areas  requiring 
additional  attention. 

In  July  1977,  the  Air 
Force  completed  implement- 
ing an  interim  depot  main- 
tenance uniform  cost  ac- 
counting system  which  in- 
cludes most  of  the  hand- 
book requirements.  The  one 
major  requirement  with 
which  Air  Force  does  not 
comply  is  the  requirenient 
to  collect  actual  hours  iden- 
tified to  individual  job  or- 
ders. DoD  has  waived  this 
requirement  for  the  Air 
Force  pending  Air  Force 
acquisition  and  implemen- 
tation of  an  on-line  auto- 
mated source  data  collection 
system. 

The  Air  Force  supports 
the  principles  of  uniform 
cost  accounting  and  will 
continue  its  efforts  to  com- 
ply with  DoD  7220.29H. 

F.  M.  ROGERS 
General,  USAF 
Commander,  Air  Force 
Logistics  Command 


Mr.  Jivatode,  the  author, 
replied: 

The  article  was  not  in- 
tended to  imply  the  Air 
Force  had  not  implemented 
any  portion  of  the  DoD  Uni- 
form Depot  Maintenance 
Cost  Accounting  System. 
The  statement  was  that  the 
Air  Force  plans  full  imple- 
mentation by  October  1979. 
The  other  Military  Services 
had  scheduled  full  imple- 
mentation for  October  1, 
1976.  The  author  was  in- 
strumental in  initiating  the 
Defense  Audit  Service  re- 
view to  determine  the  de- 
gree of  implementation  ac- 
tually experienced  by  all 
Services.  At  the  time  the 
article  was  submitted  for 
publication  the  results  of 
the  audit  ivere  not  available 
nor  had  the  Air  Force  im- 
plemented its  interim  sys- 
tem. 

Fidl  implementation  of 
this  system  will  he  largely 
dependent  on  the  attitude 
toward  it  and  emphasis 
placed  upon  it  by  the  senior 
officials  involved.  General 
Rogers’  expressed  interest 
and  support  is  demonstra- 
tive of  what  is  required  for 
successful  implementation. 
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Hot  Reading  for  a Cold  Winter’s  Day 


It  will  come  as  no  surprise  to  regular  readers  of  the 
DMJ  that  this  issue  marks  the  passing  of  an  era.  As 
heralded,  the  Journal  will  become  a bimonthly  publica- 
tion with  its  January  1978  issue.  Still,  excluding  the 
change  in  publishing  frequency,  will  anything  really  be 
different?  We  think  so. 

In  our  next  issue,  contributors  will  range  from  a Navy 
lieutenant  to  an  industry  vice  president,  from  an  Army 
systems  manager  to  a United  States  Senator,  and  from 
a group  of  professors  to  a rear  admiral.  And  the  subject 
matter  is  as  diverse  as  the  experience  of  the  authors. 
But  the  proof  of  the  talent  is  in  the  articles’  reading. 

While  jokes  about  a changing  role  for  women  in  the 
military  abound,  we  all  know  that  this  is  no  laughing 
matter.  Recently  the  quality  of  our  recruits  has  been 
seriously  questioned,  and  one  of  the  most  viable  options 
for  allaying  this  problem  seems  to  be  utilizing  the  fair 
sex  more  effectively.  Reporting  on  a case  study  of  Navy 
tugs,  Lt.  John  E.  Kane  analyzes  some  of  the  fundamental 
managerial  problems  involved  when  women  are  assigned 
to  a traditionally  all-male  environment.  Although  the 
problems  are  essentially  predictable,  some  of  the  solu- 
tions may  make  you  reconsider  how  firmly  grounded 
your  own  management  and  leadership  skills  are. 

In  1973,  a Presidential  Memorandum  made  manage- 
ment by  objectives  the  official  administrative  policy  of 
the  federal  government.  However,  reported  results  about 
the  effectiveness  of  MBO  have  been  spotty  at  best. 
Accordingly,  an  interdisciplinary  research  team  of  pro- 
fessors from  Virginia  Tech  was  hired  to  ascertain  if 
MBO  is  a feasible  technique  for  directing  the  scientific 
work  in  a typical  research  and  development  facility.  They 
discovered  that,  while  most  federal  agencies  use  MBO 
primarily  to  control  middle  management’s  execution  of 
programs,  enormous  advantages  can  be  gained  from  a 
program-  and  people-oriented  MBO  which  permeates  the 
entire  organization.  Solid  research  by  a group  of  true 
professionals  earmarks  this  enlightening  reading. 

An  equally  worthwhile  message  can  be  found  in  Col. 
John  C.  Bahnsen’s  candid  article  on  the  Army’s  total 
system  development  approach.  Using  a 1976  Army  study 
critical  of  the  service’s  integration  of  personnel,  logis- 
tics, and  training  requirements  on  anti-armor  weapon 
systems,  our  outspoken  author  offers  a logical  alternative 
and  discusses  simulation  as  a key  approach  to  improv- 
ing the  Army’s  readiness  posture  in  a cost-effective 
manner. 


The  word  from  the  pressure-packed  world  of  Capitol 
Hill  is  that  DoD  has  indeed  been  a leader  in  manage- 
ment. Colorado’s  Senator  Gary  Hart,  an  industrious  mem- 
ber of  the  Armed  Services  Committee,  has  proposed 
legislation  that  would  make  life  cycle  costing  an  every- 
day term  in  today’s  energy-conscious  America.  In  his 
article.  Senator  Hart  calls  on  the  Defense  Department  to 
share  its  wealth  of  knowledge  and  experience  with  the 
public  and  other  federal  agencies.  Read  the  article  and 
you’ll  agree:  it’s  time  to  heed  the  call. 

This  is  but  a small  sampling  of  the  wide  variety  of 
subjects  covered  in  the  inaugural  bimonthly  issue.  There 
will  also  be  articles  on  what  the  Navy  has  learned  in  its 
efforts  to  incorporate  zero-base  budgeting,  and  a con- 
tractor’s successful  method  for  operating  a management 
system  dedicated  to  the  maintenance  of  mature  and  out- 
of-production  military  hardware.  Plus  our  standard  de- 
partments on  short  informational  items,  of  course.  There 
should  be  something  for  everyone. 

So  next  January,  as  you  warm  your  chilled  body  before 
the  family  fireplace  (or  huddle  in  your  office  beseeching 
the  gods  of  the  heating  vent),  warm  your  mind  with  a 
few  hot  management  articles.  Pass  the  word:  you’ll  find 
them  in  the  new  Defense  Management  Journal. 
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ACQUISITION 


“Going  Commercial”  With  the  Commercial 
Commodity  Acquisition  Program 

hy  Cdr.  Cornelius  E.  Mandel,  Jr.,  USN 33  July 

Balancing  Readiness  and  Performance 

by  Adm.  Frederick  H.  Michaelis,  USN 46  Jan. 

Upgrading  Manufacturing  Technology 

by  Gen.  William  J.  Evans,  USAF  46  Jan. 

Early  Initiatives  in  the  Acquisition  Process 

by  Gen.  F.  Michael  Rogers,  USAF 47  Jan. 


AVIONICS 

Operational  Influences  on  Avionics  Reli- 
ability 

by  George  Kern 17  Oct. 

CONTRACTING 


Contractor  Investment  in  Defense  Industry 
— Where  Is  it? 

by  Dale  R.  Babione 9 Apr. 

A Contractor’s  View  of  System  Support 

by  Harold  Bayer 22  July 

Interim  Contractor  Support  as  a Prelude  to 
Organic  Maintenance 

by  Maj.  Robert  A.  Gulbrandsen,  USAF  29  July 


Government-Contractor  Adversarial  Rela- 
tionships 

by  Lt.  Col.  Michael  B.  Patterson,  USAF-  57  July 

Government  Engineers  in  a Major  Prime 
Contractor’s  Plant:  How  Real  is  the 
Need? 


by  Lt.  Col.  Edward  C.  Solomon,  USAF  63  July 

Taking  a Look  at  the  Request  for  Proposal 

by  Cecil  V.  Hynes 26  Oct. 


COSTS 

Capitalizing  on  Cost-Reducing  Opportunities 


by  Lt.  Col.  Grant  E.  Gabel,  USAF 24  Jan. 

Manufacturing  Technology  + Teamwork  = 

Economy 

by  Charles  P.  Downer 28  Apr. 

Tracking  the  Costs  of  Depot  Maintenance 

by  John  R.  Jivatode  10  July 

On  the  Effects  of  Modularity  on  Operating 
and  Support  Costs 

by  Rustle  Em  Jenay 32  Oct. 


FEEDBACK 

Results  of  the  Recently  Conducted  Journal 
Readership  Survey 

by  The  Editorial  Staff 65  Apr. 


Title  P.  Mo. 

LIFE  CYCLE  COSTS 

Let’s  Get  Serious  About  Life  Cycle  Costs 

by  Col.  Elbridge  P.  Eaton,  USAF 2 Jan. 

The  Yesterday,  Today,  and  Tomorrow  of 
Integrated  Logistics  Support 

by  Maj.  Ned  H.  Criscimagna,  USAF 59  Oct. 


LOGISTICS 

Common  Sense  Compromises 

by  Gen.  John  R.  Deane,  Jr.,  USA 45  Jan. 

Project  Hand-Off:  Fielding  Confidence  in 
Equipment 

by  Lt.  Gen.  Eugene  J.  D’ Ambrosia,  USA_  12  Jan. 

MAINTENANCE 


The  Army  Departs  from  Training  Tradition 

by  Lt.  Col.  Ross  S.  Buchan  and 

Maj.  Ray  Knutson,  USA 33  Jan. 

“It  Isn’t  the  Cost;  It’s  the  Upkeep” 

by  Joseph  G.  Turke  2 July 

Tracking  the  Costs  of  Depot  Maintenance 

by  John  R.  Jivatode 10  July 

The  Analytical  Maintenance  Program:  No 
More  “Maintenance  As  Usual” 

by  Rear  Adm.  Cyril  T.  Faulders,  USN 15  July 


MANAGEMENT 


Reliability  Success  Via  Total  Management 
Involvement 

by  Donald  M.  Fradette  and 

John  W.  McFarland 59  Jan. 

A Simple  Tool  for  the  Working  Manager 

by  Lt.  Col.  Gerald  E.  Rutledge,  USA 68  Jan. 

Challenging  DoD  Managers  to  Improve 
Internal  Productivity 

by  Pete  G.  Poulos 34  Apr. 

Labor-Management  Councils:  No  One 

Loses ! 

by  George  H.  Kuper  59  Apr. 

An  Innovative  Approach  to  the  Dilemma 
of  Public  Management 

by  Douglas  M.  Johnston 73  Apr. 

Facilitating  Total  Application  of  Experi- 
ence to  Work 

by  Ralph  C.  Hook 81  Apr. 

Conflict  in  Organizations:  Good  or  Bad? 
by  Lt.  Col.  Russell  Pierre,  Jr.,  USA  and 

Jerome  G.  Peppers,  Jr. 47  Oct. 

Reaching  a Supply  Efficiency  Peak 
Through  Selective  Management 

by  Stephen  A.  Stromp,  Jr. 54  Oct. 
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P. 


P. 


Mo. 


Mo. 

PROCUREMENT 

A University  Without  Walls 

by  John  H.  Kunsemiller 56  Jan. 

PRODUCTIVITY 

Productivity:  A Defense  Department  Per- 
spective 


by  Richard  J.  Power 2 Apr. 

Productivity  Growth  Through  Work  Meas- 
urement 

by  Col.  Michael  A.  Nassr,  USAF 16  Apr. 

Orthodox  Job  Enrichment:  A Common 
Sense  Approach  to  People  at  Work 

by  Frederick  I.  Herzberg 21  Apr. 

Challenging  DoD  Managers  to  Improve  In- 
ternal Productivity 

by  Pete  G.  Poulos 34  Apr. 

Using  Computers  to  Improve  Manufactur- 
ing Productivity 

by  Dennis  E.  Wisnosky 41  Apr. 

Computer-Aided  Manufacturing:  Another 
Industrial  Revolution? 

by  John  H.  Schulz 51  Apr. 


RELIABILITY 

Reliability  Success  Via  Total  Management 


Involvement 

by  Donald  M.  Fradette  and 

John  W.  McFarland 59  Jan. 

Operational  Influences  on  Avionics  Reli- 
ability 

by  George  Kem 17  Oct. 


RESEARCH  AND  DEVELOPMENT 

The  What’s  and  Why’s  of  Independent 
Research  and  Development 

by  Karl  G.  Harr,  Jr. 77  Apr. 

Tapping  Organizational  Functioning  in  the 
Federal  Sector  Through  Systematic 
Survey  Research 

by  Stanley  L.  Cohen  and  John  R.  Turney  44  July 
Advanced  Development  Technology  Pro- 
grams: Do  We  Need  a New  Funding 
Approach? 

by  Lt.  Col.  John  W.  Barlow,  USAF 49  July 

SPECIFICATIONS  AND  STANDARDS 

Tailoring  Specifications  and  Standards  to 
Today’s  Needs 

by  Lester  Fox  and  Jeffrey  S.  Allan 10  Oct. 


Title 

STATISTICS 

When  Managing — Think  Statistics 

by  Rose  Glubin 48  Jan. 


TECHNOLOGY 

The  Use  of  Simulators:  A New  Thrust  in 
the  Technology  Base 

by  John  L.  Allen 30  Jan. 

The  Guided  Missile  Frigate:  New  Technol- 
ogy for  a New  Era 

by  Stephen  P.  Light 38  Jan. 

Upgrading  Manufacturing  Technology 

by  Gen.  William  J.  Evans,  USAF 46  Jan. 

Manufacturing  Technology  + Teamwork 
Economy 

by  Charles  P.  Downer 28  Apr. 

Using  Computers  to  Improve  Manufactur- 
ing Productivity 

by  Dennis  E.  Wisnosky 41  Apr. 

Computer-Aided  Manufacturing:  Another 
Industrial  Revolution? 

by  John  H.  Schulz 51  Apr. 


TEST  AND  EVALUATION 


A Sophisticated  Test  System  to  Support 
Modern  Weaponry 

by  T.  W.  Taylor  and  E.  F.  Wares 16  Jan. 

Reducing  Risk  Through  Testing 

by  Rear  Adm.  L.  S.  Kollmorgen,  USN 2 Oct. 

Relating  Promised  Performance  to  Military 
Worth:  An  Evaluating  Mechanism 
by  Michael  L.  Hays  and 

Michael  F.  O’Connor 36  Oct. 


WEAPON  SYSTEMS 

A Sophisticated  Test  System  to  Support 


Modern  Weaponry 

by  T.  W.  Taylor  and  E.  F.  Wares 16  Jan. 

The  Guided  Missile  Frigate:  New  Technol- 
ogy for  a New  Era 

by  Stephen  P.  Light 38  Jan. 

Common  Sense  Compromises 

by  Gen.  John  R.  Deane,  Jr.,  USA 45  Jan. 

Balancing  Readiness  and  Performance 

by  Adm.  Frederick  H.  Michaelis,  USN  46  Jan. 
Upgrading  Manufacturing  Technology 

by  Gen.  William  J.  Evans,  USAF 46  Jan. 

Early  Initiatives  in  the  Acquisition  Process 

by  Gen.  F.  Michael  Rogers,  USAF 47  Jan. 
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CALENDAR 

Event 

Date 

Place 

Contact 

Critical  Path  Method: 
Project  Planning  and 
Control 

Oct.  29-Dec.  17 
(Saturday  mornings) 

Washington,  DC 

Director,  Continuing 
Engineering  Education 
George  Washington  University 
Washington,  DC  20007 
(202)  676-6106 

Government-wide  Require- 
ments for  Performance 
Measurement  Systems — 
for  contractors  and 
managers 

Nov.  2-4 

Washington,  DC 

AIIE  Seminars 
P.O.  Box  25116 
Los  Angeles,  CA  90025 
(213)  450-0500 

Seminar:  Application  and 
Tailoring  of  Specifica- 
tions and  Standards 

Nov.  8-10 

Monterey,  CA 

T.  L.  Golmis,  Industry 
Coordinator 

Hughes  Aircraft  Company 

P.O.  Box  3310 

Fullerton,  CA  92634 

(714)  871-3232 

Sam  P.  Miller,  Government 

Coordinator 

DMSSO,  Cameron  Station 
Alexandria,  VA  22314 
(202)  274-6728 

Market  Forecast:  Five 
years  and  Beyond — for 
aerospace,  defense,  and 
electronics  managers 

Nov.  28-29 

Los  Angeles,  CA 

State  of  the  Art  Seminars,  D/MF 
P.O.  Box  92932 
Los  Angeles,  CA  90009 
(213)  437-7465 

Middle  Management 
Assessment  Center — to 
identify  and  develop 
skills 

Nov.  28-30 
Feb.  15-17 

Washington,  DC 

Leslie  Bobrowski/ Karen  Niles 
USDA  Graduate  School 
529  14th  St.,  N.W. 
Washington,  DC  20045 
(202)  447-3247 

Federal  Procurement 

Dec.  5-6 

Los  Angeles,  CA 

Heidi  Kaplan,  Dept.  14NR 

Seminar — for  contractors 

Feb.  7-8 

New  York,  NY 

New  York  Management  Center 
360  Lexington  Ave. 

New  York,  NY  10017 
(212)  953-7262 

Configuration  Management 
of  Software  Programs 

Jan.  23-25 

Washington,  DC 

see  George  Washington 
University  above 

Zero-Base  Budgeting 
Seminar — for  managers 
and  analysts 

Feb.  1-3 

Ann  Arbor,  Ml 

Division  of  Management 
Education 

The  University  of  Michigan 
1735  Washtenaw  Avenue 
Ann  Arbor,  Ml  48109 
(313)  764-1817 
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